-- Logs begin at Fri 2022-09-09 22:22:20 UTC, end at Fri 2022-09-09 23:20:42 UTC. -- Sep 09 22:47:57.005649 dv-cinder-852211-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 22:47:57.013387 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 22:47:57.015383 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 22:47:57 2022] *** Sep 09 22:47:57.015383 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 22:47:57.015383 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 22:47:57.015383 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: nodename: dv-cinder-852211-14-iscsi Sep 09 22:47:57.015383 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: machine: x86_64 Sep 09 22:47:57.015383 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: clock source: unix Sep 09 22:47:57.015383 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: pcre jit disabled Sep 09 22:47:57.015849 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: detected number of CPU cores: 4 Sep 09 22:47:57.015849 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: current working directory: / Sep 09 22:47:57.015849 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: detected binary path: /usr/bin/uwsgi-core Sep 09 22:47:57.015849 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: your processes number limit is 47680 Sep 09 22:47:57.015849 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: your memory page size is 4096 bytes Sep 09 22:47:57.015849 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: detected max file descriptor number: 2048 Sep 09 22:47:57.015849 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: lock engine: pthread robust mutexes Sep 09 22:47:57.017912 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: thunder lock: enabled Sep 09 22:47:57.017912 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 22:47:57.017912 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 22:47:57.055835 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: Python main interpreter initialized at 0x55f34ff009c0 Sep 09 22:47:57.055835 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: python threads support enabled Sep 09 22:47:57.055835 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: your server socket listen backlog is limited to 100 connections Sep 09 22:47:57.055835 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: your mercy for graceful operations on workers is 90 seconds Sep 09 22:47:57.056222 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 22:47:57.056303 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: *** Operational MODE: preforking *** Sep 09 22:47:57.056515 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: *** uWSGI is running in multiple interpreter mode *** Sep 09 22:47:57.056515 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: spawned uWSGI master process (pid: 81392) Sep 09 22:47:57.056720 dv-cinder-852211-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 22:47:57.057870 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: spawned uWSGI worker 1 (pid: 81403, cores: 1) Sep 09 22:47:57.057870 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: spawned uWSGI worker 2 (pid: 81404, cores: 1) Sep 09 22:47:57.057870 dv-cinder-852211-14-iscsi devstack@keystone.service[81392]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 22:48:01.336085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 22:48:01.336085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Sep 09 22:48:01.336884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 22:48:01.336884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Sep 09 22:48:01.337278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 22:48:01.337278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: from cryptography.utils import int_from_bytes Sep 09 22:48:01.337731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 22:48:01.337731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: from cryptography.utils import int_from_bytes Sep 09 22:48:01.385749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 22:48:01.385935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 22:48:01.386127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] command line args: [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 22:48:01.386393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 22:48:01.386569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ================================================================================ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 22:48:01.387009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.387122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 22:48:01.387253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_dir = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.387331 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 22:48:01.387441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.387539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] command line args: [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 22:48:01.387651 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_source = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.387781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 22:48:01.387859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] debug = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.387961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ================================================================================ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 22:48:01.388045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.388266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] admin_token = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.388398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.388504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config_dir = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.388606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.388701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.388800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.388900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] config_source = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] debug = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_config_append = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_dir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.389999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_config_append = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_dir = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.390918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.391907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392235 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.392921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] publish_errors = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.393909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.394959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] publish_errors = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] standard_threads = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.395924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_journal = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_json = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_stderr = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] standard_threads = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_syslog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.396896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.397025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.397107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.397183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.397261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_journal = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.397372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.397450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_json = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.397569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_stderr = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.397644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.397747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] use_syslog = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.397835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.397981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 22:48:01.398064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.398361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.398439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.398508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.398585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.398715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.398792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.398957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399285 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.399937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.400968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.401929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.402089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.402724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.402857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.402932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.403888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404123 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.404985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.405936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.406926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.407872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.408938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.409976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.410925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411264 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.411981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.412916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.413881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.414899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.415868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416264 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416814 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.416956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.417034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.417186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.417265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.417418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.417504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.417662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.417738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.418004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.418118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.418768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.418906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.419007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.419185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.419263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.419447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.419566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.419735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.420957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.421979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.422918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.423900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.external = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.password = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.424884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.external = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.token = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.425991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.password = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] auth.token = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.426950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.427938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.428992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.429947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.430937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.431929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.432880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.433843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.434965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.435945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.436886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.437908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438639 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.438872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.439991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440069 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.440932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.441953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.442888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.443920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.444876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.445862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.446985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.447959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.448995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.449668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.449753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.449910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.449986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.450939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.451917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.452865 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.453968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.454999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.455156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.455242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.455395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.455472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.455663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.455743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.455909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.456942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.457902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.458847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.459833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.460973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.461945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.462108 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.462185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.462358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.462446 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.462609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.462688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.462859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.463901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.464065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.464142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.464298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.464376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.464530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.464616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.464762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.465372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.465578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.465657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.465822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.465909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.466995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467814 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.467966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.468936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.469961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470127 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.470935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.471955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.472893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.473840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.474950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475680 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.475852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 22:48:01.475938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.476150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.476382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.476625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.476860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 22:48:01.477033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 22:48:01.504624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.504856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.504955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.505086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.505170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.505401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.513930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.514302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.542140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.542369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.542519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.542754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.595273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.595660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.644020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.644252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.644798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.645039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.800240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.800478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.800704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.800867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.800977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.801107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.801191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.801333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.801415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.801556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.801775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.801993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.824165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.824388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.824614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.824720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.824840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.824923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.825963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.836608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.836968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.849790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.849973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850199 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.850936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.851073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.851154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.851297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 22:48:01.867830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.868040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 22:48:01.868384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.868585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 22:48:01.868772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.868868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.868987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 22:48:01.869112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.869331 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.869412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.869532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 22:48:01.869614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.869807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.870071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.870334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.870595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.879728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.880011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.880286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.880575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.880747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.881032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.881307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.881594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.887591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.888681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.890502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.890789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.891077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.891360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.891614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.891699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.891904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.891986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.892193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.892274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.892463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.892550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.892732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.892812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.893004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.893084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.893272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.893354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.893546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.893823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.894100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.894370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.916354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.917501 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.917821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.918154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.919034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.919322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.923098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.923384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.923692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.925250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.925536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.925821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.930023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.930313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.932205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.932495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.936427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.936724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.937016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.937306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.937587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.937866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.938140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.938416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.938622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.938921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.939214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.939529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.939818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.940100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.940373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.940644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.953991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.954280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.956178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.956461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.958127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.960292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.961397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.961695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.963471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.963794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.966421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.968486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.969580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.971648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.972847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.973147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.973435 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.973722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.974004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.974293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.974887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.975190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.975477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.975787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.976068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.976348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.990125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.990412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.990679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.990941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.992218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:01.992513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.992787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.993053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.998342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:01.999908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.000519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.001442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.001725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.002066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.003641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.003923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.004459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.004747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.005038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.005321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.005711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.006726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.007016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.007303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.007627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.008034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.016924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.017211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.017491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.017772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.018152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.018441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.018720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.019272 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.019590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.019881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.020163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.020549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.020839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.021131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.034656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.037097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.037612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.040064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.040526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.042005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.043036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.043548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.043834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.044232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.044543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.046277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.046560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.046968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.050868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.052356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.053435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.054905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.055243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.055661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.055952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.056233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.057830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.058232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.059048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.059329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.064598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.064891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.065190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.067172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.067466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.067772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.072086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.072372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.072656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.072938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.073227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.073498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.073769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.074052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.074857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.075125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.075260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 22:48:02.075390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.075552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.075796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.075929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.076056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.076190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.076314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.076439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.076605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.076884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.077175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.077457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.077738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.078021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.078286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.078552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.078821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 22:48:02.110240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f34ff009c0 pid: 81403 (default app) Sep 09 22:48:02.112769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f34ff009c0 pid: 81404 (default app) Sep 09 22:48:02.113519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95039c-f7b2-449b-ab69-12d3c14c5caf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:02.113806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95039c-f7b2-449b-ab69-12d3c14c5caf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:02.114079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95039c-f7b2-449b-ab69-12d3c14c5caf None None] PATH_INFO: `/v3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:02.118267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Fri Sep 9 22:48:02 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:08.845184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf553df-9c34-41be-a2f6-ad14908b5145 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:08.845691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf553df-9c34-41be-a2f6-ad14908b5145 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:08.845793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf553df-9c34-41be-a2f6-ad14908b5145 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:08.849969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:08.855755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:08.856292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:08.856292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:08.926581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: INFO dbcounter [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] Registered counter for database keystone Sep 09 22:48:08.945377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG oslo_db.sqlalchemy.engines [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81403) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 22:48:08.952052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writer thread running {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 22:48:09.131865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81403) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 22:48:09.132268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81403) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 22:48:09.190110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:09.191771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:09.319819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 22:48:09.756513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf553df-9c34-41be-a2f6-ad14908b5145 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:09.825244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: INFO dbcounter [None req-6cf553df-9c34-41be-a2f6-ad14908b5145 None None] Registered counter for database keystone Sep 09 22:48:09.846538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG oslo_db.sqlalchemy.engines [None req-6cf553df-9c34-41be-a2f6-ad14908b5145 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81404) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 22:48:09.852756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writer thread running {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 22:48:09.969347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf553df-9c34-41be-a2f6-ad14908b5145 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:09.973530 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeeb0c-bd8f-4b87-aa1d-17482330c41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:09.974154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeeb0c-bd8f-4b87-aa1d-17482330c41d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:09.974432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeeb0c-bd8f-4b87-aa1d-17482330c41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:09.974700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeeb0c-bd8f-4b87-aa1d-17482330c41d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:09.982727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeeb0c-bd8f-4b87-aa1d-17482330c41d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:10.034196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING keystone.common.rbac_enforcer.enforcer [None req-97eeeb0c-bd8f-4b87-aa1d-17482330c41d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 22:48:10.034538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeeb0c-bd8f-4b87-aa1d-17482330c41d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:10.043605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Sep 9 22:48:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 22:48:10.049954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:10.067061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99952b-a603-4273-84a3-88cdc4b12e7a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:10.071417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d0bd6-a74b-494d-95a4-f4df16c0301b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:10.072547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0bd6-a74b-494d-95a4-f4df16c0301b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:10.072827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0bd6-a74b-494d-95a4-f4df16c0301b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:10.073105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0bd6-a74b-494d-95a4-f4df16c0301b admin admin] PATH_INFO: `/v3/projects/8675cee43cc84599b492f8c81300cc82` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:10.075301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d0bd6-a74b-494d-95a4-f4df16c0301b admin admin] RBAC: Authorizing `identity:get_project(project_id=8675cee43cc84599b492f8c81300cc82)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:10.127366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING keystone.common.rbac_enforcer.enforcer [None req-217d0bd6-a74b-494d-95a4-f4df16c0301b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 22:48:10.127730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d0bd6-a74b-494d-95a4-f4df16c0301b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:10.132203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Fri Sep 9 22:48:10 2022] GET /identity/v3/projects/8675cee43cc84599b492f8c81300cc82 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:12.228031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88b8b-e1aa-4582-873d-8053ad798850 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:12.228599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88b8b-e1aa-4582-873d-8053ad798850 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:12.228599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88b8b-e1aa-4582-873d-8053ad798850 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:12.233725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:12.239329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd52b7-7366-47d9-8fb0-5b0eed141b6d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:12.239795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd52b7-7366-47d9-8fb0-5b0eed141b6d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:12.239917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd52b7-7366-47d9-8fb0-5b0eed141b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:12.260154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c6bd52b7-7366-47d9-8fb0-5b0eed141b6d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:12.261668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c6bd52b7-7366-47d9-8fb0-5b0eed141b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:12.269344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 22:48:12.531360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b88b8b-e1aa-4582-873d-8053ad798850 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:12.543880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b88b8b-e1aa-4582-873d-8053ad798850 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:12.548328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8853632-cb58-4c0d-8215-98bb87ee7f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:12.548923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8853632-cb58-4c0d-8215-98bb87ee7f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:12.549199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8853632-cb58-4c0d-8215-98bb87ee7f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:12.549467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8853632-cb58-4c0d-8215-98bb87ee7f53 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:12.556938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8853632-cb58-4c0d-8215-98bb87ee7f53 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:12.558745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8853632-cb58-4c0d-8215-98bb87ee7f53 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:12.567255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Sep 9 22:48:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 22:48:12.572393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd52b7-7366-47d9-8fb0-5b0eed141b6d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:12.581292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd52b7-7366-47d9-8fb0-5b0eed141b6d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:12.585552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d17bc7-7d44-491b-9ed1-cedaf9a7ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:12.586166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d17bc7-7d44-491b-9ed1-cedaf9a7ded9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:12.586445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d17bc7-7d44-491b-9ed1-cedaf9a7ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:12.586716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d17bc7-7d44-491b-9ed1-cedaf9a7ded9 admin admin] PATH_INFO: `/v3/users/fa41a2a2137d4147a83f9f036ba03916` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:12.588037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d17bc7-7d44-491b-9ed1-cedaf9a7ded9 admin admin] RBAC: Authorizing `identity:get_user(user_id=fa41a2a2137d4147a83f9f036ba03916)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:12.590393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d17bc7-7d44-491b-9ed1-cedaf9a7ded9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:12.591912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Fri Sep 9 22:48:12 2022] GET /identity/v3/users/fa41a2a2137d4147a83f9f036ba03916 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:14.778454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c229ea7-3386-430a-b01f-7dbcea1babf0 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:14.778999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c229ea7-3386-430a-b01f-7dbcea1babf0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:14.778999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c229ea7-3386-430a-b01f-7dbcea1babf0 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:14.780363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:14.785685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaf163-05a8-4586-969a-77f7edad7555 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:14.786109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaf163-05a8-4586-969a-77f7edad7555 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:14.786360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaf163-05a8-4586-969a-77f7edad7555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:14.804432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3bbaf163-05a8-4586-969a-77f7edad7555 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:14.805612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3bbaf163-05a8-4586-969a-77f7edad7555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:14.868955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67447e84-18af-4d95-b1e2-4da3da9f1a6c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:14.869237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67447e84-18af-4d95-b1e2-4da3da9f1a6c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:14.869514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67447e84-18af-4d95-b1e2-4da3da9f1a6c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:14.871433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:14.876428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:14.876712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:14.876983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:14.888546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:14.919578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG passlib.handlers.bcrypt [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81404) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 22:48:14.920228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG passlib.handlers.bcrypt [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81404) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 22:48:14.969364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:14.970939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:15.068566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:15.160044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaf163-05a8-4586-969a-77f7edad7555 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.183362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:15.262814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaf163-05a8-4586-969a-77f7edad7555 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.296188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.296812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.297088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.297359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.308723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:15.317495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:15.318541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:15.319633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:15.326545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.336278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.423045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:15.433484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7c97-1379-4600-b964-febc808d11f6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.435840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:15.441345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca482ab-e7c8-44e4-adff-091332e7b4b2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.472481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.473348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.473633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.473910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.474766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:15.475129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.475229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:15.475761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.476043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.476313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.476613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:15.489055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:15.494646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.543124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:15.544630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:15.592483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c76f7b-b73f-4f7f-8b76-c8803b41372d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.622508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:15.627679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.628533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.628819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.629096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] PATH_INFO: `/v3/users/fa41a2a2137d4147a83f9f036ba03916` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.630574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41a2a2137d4147a83f9f036ba03916)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:15.632065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:15.633634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 9/16] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/users/fa41a2a2137d4147a83f9f036ba03916 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:15.639102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.641669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 8/17] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 22:48:15.646907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.674368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:15.758424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ac3c0-1056-48e3-a632-b7d0925a73b9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.758709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e94157-d0f8-478c-a87e-5fddb76baad5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.791783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.792267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.792812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.792812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.792964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.793028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.793098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.793358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.794454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:15.796177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:15.797725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:15.803353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.803469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:15.804944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:48:15.821375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:15.822371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:48:15.823367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:48:15.823367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:48:15.823367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:15.823367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:48:15.828562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:15.862266 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:15.903768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a985b2-c851-4de4-8e85-641022d033ff None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.905264 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:15.927657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1185eb-9b1f-47e3-9ce6-a64eac039e0d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:15.937336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2d783-c70f-40b1-be74-6ac2c6036a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.937939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2d783-c70f-40b1-be74-6ac2c6036a70 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.938219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2d783-c70f-40b1-be74-6ac2c6036a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.938493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2d783-c70f-40b1-be74-6ac2c6036a70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.939215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2d783-c70f-40b1-be74-6ac2c6036a70 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.domain.id=default, user.id=fa41a2a2137d4147a83f9f036ba03916) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:15.939541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2d783-c70f-40b1-be74-6ac2c6036a70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:15.941030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2d783-c70f-40b1-be74-6ac2c6036a70 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:15.949935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/role_assignments?user.id=fa41a2a2137d4147a83f9f036ba03916&scope.domain.id=default&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:15.967613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea4d5d-8454-480d-a7b8-61208ca7b4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:15.968348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4d5d-8454-480d-a7b8-61208ca7b4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:15.968698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4d5d-8454-480d-a7b8-61208ca7b4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:15.969054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4d5d-8454-480d-a7b8-61208ca7b4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:15.969771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea4d5d-8454-480d-a7b8-61208ca7b4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:15.970141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea4d5d-8454-480d-a7b8-61208ca7b4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:15.971626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea4d5d-8454-480d-a7b8-61208ca7b4b4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:15.990879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:18.347826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014107ed-a200-46aa-b1d4-6b92d284a8ea None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:18.348844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014107ed-a200-46aa-b1d4-6b92d284a8ea None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:18.349335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014107ed-a200-46aa-b1d4-6b92d284a8ea None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:18.350890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:18.356679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96945-b0cd-423d-8809-9ae15fef1cfd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:18.357540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96945-b0cd-423d-8809-9ae15fef1cfd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:18.358028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96945-b0cd-423d-8809-9ae15fef1cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:18.376646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803af6b9-2c03-46fc-b851-0e526351b5ec None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:18.377853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803af6b9-2c03-46fc-b851-0e526351b5ec None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:18.378333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803af6b9-2c03-46fc-b851-0e526351b5ec None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:18.380227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 13/23] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:18.383033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a5a96945-b0cd-423d-8809-9ae15fef1cfd None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:18.384281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a5a96945-b0cd-423d-8809-9ae15fef1cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:18.397292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5cf01-a326-443a-9ea8-4ab823aa48ee None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:18.397897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5cf01-a326-443a-9ea8-4ab823aa48ee None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:18.398369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5cf01-a326-443a-9ea8-4ab823aa48ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:18.429040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d1d5cf01-a326-443a-9ea8-4ab823aa48ee None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:18.430584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d1d5cf01-a326-443a-9ea8-4ab823aa48ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:18.523270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 11/24] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:18.556855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 14/25] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:18.953552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5aa00-6a11-4935-a378-429ecc5b389a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:18.953929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5aa00-6a11-4935-a378-429ecc5b389a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:18.954121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5aa00-6a11-4935-a378-429ecc5b389a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:18.955462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 12/26] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:18.960834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d30c6-b356-423f-9703-6216131f7e59 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:18.961128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d30c6-b356-423f-9703-6216131f7e59 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:18.961409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d30c6-b356-423f-9703-6216131f7e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:18.983081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5aa00-6a11-4935-a378-429ecc5b389a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:18.987566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-045d30c6-b356-423f-9703-6216131f7e59 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:18.996933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-045d30c6-b356-423f-9703-6216131f7e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:19.035120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:19.147507 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 15/27] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:19.149938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d30c6-b356-423f-9703-6216131f7e59 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:19.190637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5aa00-6a11-4935-a378-429ecc5b389a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:19.207834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:19.261600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b52b8-7d5d-4b99-890b-744a100266f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:19.262619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b52b8-7d5d-4b99-890b-744a100266f2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:19.263100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b52b8-7d5d-4b99-890b-744a100266f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:19.263604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b52b8-7d5d-4b99-890b-744a100266f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:19.288929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d30c6-b356-423f-9703-6216131f7e59 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:19.290409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b52b8-7d5d-4b99-890b-744a100266f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:19.292198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b52b8-7d5d-4b99-890b-744a100266f2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a34b52b8-7d5d-4b99-890b-744a100266f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:48:19.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:19.320615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:48:19.321422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:48:19.321422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:48:19.321422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:19.323443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 13/28] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:48:19.327537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5b11d-27c2-4be3-9aa0-c1ea9d7b3348 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:19.327688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f588d-f607-48c3-b6ed-853f07450408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:19.327688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f588d-f607-48c3-b6ed-853f07450408 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:19.327688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f588d-f607-48c3-b6ed-853f07450408 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:19.327688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f588d-f607-48c3-b6ed-853f07450408 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:19.328621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5b11d-27c2-4be3-9aa0-c1ea9d7b3348 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:19.329033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5b11d-27c2-4be3-9aa0-c1ea9d7b3348 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:19.330797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 14/29] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:19.331958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5b11d-27c2-4be3-9aa0-c1ea9d7b3348 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:19.354976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f588d-f607-48c3-b6ed-853f07450408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fc7f588d-f607-48c3-b6ed-853f07450408 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:19.365097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:19.366155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:19.367105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:19.368986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 16/30] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:19.370140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf3987-fc0a-4042-846e-d907a1367e39 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:19.370997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf3987-fc0a-4042-846e-d907a1367e39 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:19.371477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf3987-fc0a-4042-846e-d907a1367e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:19.402979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bdcf3987-fc0a-4042-846e-d907a1367e39 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:19.404200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bdcf3987-fc0a-4042-846e-d907a1367e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:19.474427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:19.507633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 17/31] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:19.508044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5b11d-27c2-4be3-9aa0-c1ea9d7b3348 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:19.508479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf3987-fc0a-4042-846e-d907a1367e39 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:19.547397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:19.547993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:19.548281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:19.548558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:19.549210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:19.549518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:19.550923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:19.553455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:19.567349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 15/32] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:19.568709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:19.685001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffc030-2df8-4ba0-9bd6-a6f8a9dd5b23 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:19.729199 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:19.729901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:19.730206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:19.730489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:19.731197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:19.733183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:19.742746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:19.752733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf3987-fc0a-4042-846e-d907a1367e39 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:19.828851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:19.829385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:19.829659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:19.829924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:19.830657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:19.830959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:19.832365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:19.852450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 18/33] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:19.853051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:19.886937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 16/34] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 22:48:19 2022] POST /identity/v3/roles => generated 219 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:19.888419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:20.007831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:20.021952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8220ba2-beea-4137-98a3-54303a3a5d53 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:20.060571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff9c1a-f38e-4abb-8e24-b431f919e648 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:20.079960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:20.081726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.081996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.082272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] PATH_INFO: `/v3/users/fa41a2a2137d4147a83f9f036ba03916` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.086248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] RBAC: Authorizing `identity:get_user(user_id=fa41a2a2137d4147a83f9f036ba03916)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:20.091137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:20.100930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 17/35] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:19 2022] GET /identity/v3/users/fa41a2a2137d4147a83f9f036ba03916 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:20.101588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:20.140756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:20.141436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.141716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.141986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.142747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:20.144404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:20.217060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:20.264331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb47cf0-7a22-4445-8497-b60f50f5cfcc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:20.305606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46126cb-1395-4275-86b3-ace088fcf798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:20.306235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46126cb-1395-4275-86b3-ace088fcf798 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.306518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46126cb-1395-4275-86b3-ace088fcf798 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.306797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46126cb-1395-4275-86b3-ace088fcf798 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.307506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46126cb-1395-4275-86b3-ace088fcf798 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:20.308952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46126cb-1395-4275-86b3-ace088fcf798 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:20.322516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 19/36] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:48:19 2022] POST /identity/v3/projects => generated 301 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:20.323302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:20.325271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 18/37] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 22:48:20 2022] POST /identity/v3/roles => generated 225 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:20.326487 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1d240-2f1b-4d54-a2c8-22d6b548e8e9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.326990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1d240-2f1b-4d54-a2c8-22d6b548e8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.327286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1d240-2f1b-4d54-a2c8-22d6b548e8e9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.329779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 19/38] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:20.331132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab42e4-af14-4807-94f1-0d93550c92df None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.331362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab42e4-af14-4807-94f1-0d93550c92df None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.331674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab42e4-af14-4807-94f1-0d93550c92df None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.333120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 20/39] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:20.335697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a650cf-4b78-4601-919d-9b865dcd331b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.335986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a650cf-4b78-4601-919d-9b865dcd331b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.336271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a650cf-4b78-4601-919d-9b865dcd331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.362666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-59a650cf-4b78-4601-919d-9b865dcd331b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:20.365108 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-59a650cf-4b78-4601-919d-9b865dcd331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:20.449031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204672f8-e069-4512-b13c-be0383c93405 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:20.469428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 21/40] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:20.470665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa516-9efa-43d5-9289-9a72f663c7fd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.470966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa516-9efa-43d5-9289-9a72f663c7fd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.471244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa516-9efa-43d5-9289-9a72f663c7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.529122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:20.539970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:20.540993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.541448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.541897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.545216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:20.547063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:20.549006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 20/41] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:20.555161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:20.610466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a82aa516-9efa-43d5-9289-9a72f663c7fd None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:20.611645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a82aa516-9efa-43d5-9289-9a72f663c7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:20.682313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:20.688037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a607d-159a-4434-8285-37bc1b6c73ab None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:20.702717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 22/42] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:20.735898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:20.736589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.736871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.737141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] PATH_INFO: `/v3/domains/default/users/fa41a2a2137d4147a83f9f036ba03916/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.739166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fa41a2a2137d4147a83f9f036ba03916, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:20.740621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:20.754131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 21/43] 10.222.0.17 () {58 vars in 1361 bytes} [Fri Sep 9 22:48:20 2022] PUT /identity/v3/domains/default/users/fa41a2a2137d4147a83f9f036ba03916/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 22:48:20.786505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa516-9efa-43d5-9289-9a72f663c7fd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:20.897483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa516-9efa-43d5-9289-9a72f663c7fd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:20.938617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:20.939242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:20.939631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:20.939913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:20.940603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:20.942082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:20.958313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 23/44] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 22:48:20 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:20.992007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:21.062372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:21.117317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee309fa-eedd-481e-919a-c032ab0edba5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:21.154940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8d42b-7d7a-4cb8-9f1a-1b29206caf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:21.155637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8d42b-7d7a-4cb8-9f1a-1b29206caf59 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:21.155923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8d42b-7d7a-4cb8-9f1a-1b29206caf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:21.156195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8d42b-7d7a-4cb8-9f1a-1b29206caf59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:21.157319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8d42b-7d7a-4cb8-9f1a-1b29206caf59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:21.158769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8d42b-7d7a-4cb8-9f1a-1b29206caf59 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:21.160387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 22/45] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:21.166127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:21.216222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:21.272285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc54a2-4211-46d5-a6aa-a5222c4f1d9f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:21.306338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8be0f-8a1d-4a09-9b8b-9d6aa74bdeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:21.306952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8be0f-8a1d-4a09-9b8b-9d6aa74bdeaa None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:21.307236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8be0f-8a1d-4a09-9b8b-9d6aa74bdeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:21.307536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8be0f-8a1d-4a09-9b8b-9d6aa74bdeaa None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:21.308222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8be0f-8a1d-4a09-9b8b-9d6aa74bdeaa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:21.309664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8be0f-8a1d-4a09-9b8b-9d6aa74bdeaa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:21.340546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:48:21 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:48:22.904024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffbf4d-6e4b-4512-bc69-b4a40051cdb5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:22.904557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffbf4d-6e4b-4512-bc69-b4a40051cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:22.904648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffbf4d-6e4b-4512-bc69-b4a40051cdb5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:22.905925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:22.911094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c2cd2-9fb0-4672-8a32-bb901d6abb11 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:22.911484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c2cd2-9fb0-4672-8a32-bb901d6abb11 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:22.911753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c2cd2-9fb0-4672-8a32-bb901d6abb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:22.929809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cd1c2cd2-9fb0-4672-8a32-bb901d6abb11 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:22.930992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cd1c2cd2-9fb0-4672-8a32-bb901d6abb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:22.996339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:23.274422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffbf4d-6e4b-4512-bc69-b4a40051cdb5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:23.374716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffbf4d-6e4b-4512-bc69-b4a40051cdb5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:23.403438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:23.412398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753981a4-7554-40b7-8bbd-0f6bc56a2c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:23.413031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753981a4-7554-40b7-8bbd-0f6bc56a2c3b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:23.413315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753981a4-7554-40b7-8bbd-0f6bc56a2c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:23.413591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753981a4-7554-40b7-8bbd-0f6bc56a2c3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:23.424259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753981a4-7554-40b7-8bbd-0f6bc56a2c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-753981a4-7554-40b7-8bbd-0f6bc56a2c3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:23.429045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:23.430022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:23.430775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 24/49] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:23.436085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c2cd2-9fb0-4672-8a32-bb901d6abb11 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:23.461929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:23.496271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f22b6-dee0-4937-827d-fbe8a9130783 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:23.498087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f22b6-dee0-4937-827d-fbe8a9130783 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:23.499007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f22b6-dee0-4937-827d-fbe8a9130783 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:23.500409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 25/50] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:23.507574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301dfe1-9f34-46ad-b5f8-b0754eb8d49c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:23.507862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301dfe1-9f34-46ad-b5f8-b0754eb8d49c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:23.508137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301dfe1-9f34-46ad-b5f8-b0754eb8d49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:23.530765 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b301dfe1-9f34-46ad-b5f8-b0754eb8d49c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:23.532244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b301dfe1-9f34-46ad-b5f8-b0754eb8d49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:23.544213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c2cd2-9fb0-4672-8a32-bb901d6abb11 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:23.580129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db791-7af3-4291-9218-d714e5057422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:23.580770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db791-7af3-4291-9218-d714e5057422 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:23.581024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db791-7af3-4291-9218-d714e5057422 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:23.581296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db791-7af3-4291-9218-d714e5057422 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:23.581957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247db791-7af3-4291-9218-d714e5057422 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:23.582262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247db791-7af3-4291-9218-d714e5057422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:23.583681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247db791-7af3-4291-9218-d714e5057422 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:23.596078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 26/51] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:23.599944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 26/52] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:23.601945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db791-7af3-4291-9218-d714e5057422 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:23.685578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:23.697383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db791-7af3-4291-9218-d714e5057422 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:23.730059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:23.730698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:23.730975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:23.731247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] PATH_INFO: `/v3/users/fa41a2a2137d4147a83f9f036ba03916` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:23.732444 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41a2a2137d4147a83f9f036ba03916)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:23.733884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:23.735446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 27/53] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:23 2022] GET /identity/v3/users/fa41a2a2137d4147a83f9f036ba03916 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:23.741013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301dfe1-9f34-46ad-b5f8-b0754eb8d49c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:23.840183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301dfe1-9f34-46ad-b5f8-b0754eb8d49c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:23.856769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:23.868919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:23.874454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:23.875092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:23.875373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:23.875679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:23.876800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:23.878247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:23.879832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 27/54] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:23.885316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:23.971018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca48f79-85e7-474b-9b20-3c8da32dd279 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:23.982058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7eaf8-3e1e-4e2a-9f26-55fbcd2a61b5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:24.004442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:24.005074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:24.005357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:24.005631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:24.006748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:24.008225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:24.009781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 28/55] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:24.014909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8548ca0-8863-41f9-bdfe-c254dd1abaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:24.015535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8548ca0-8863-41f9-bdfe-c254dd1abaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:24.015663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:24.015822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8548ca0-8863-41f9-bdfe-c254dd1abaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:24.016098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8548ca0-8863-41f9-bdfe-c254dd1abaa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:24.016823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8548ca0-8863-41f9-bdfe-c254dd1abaa7 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.domain.id=default, user.id=fa41a2a2137d4147a83f9f036ba03916) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:24.017126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8548ca0-8863-41f9-bdfe-c254dd1abaa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:24.018546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8548ca0-8863-41f9-bdfe-c254dd1abaa7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:24.035533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 28/56] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Sep 9 22:48:23 2022] GET /identity/v3/role_assignments?user.id=fa41a2a2137d4147a83f9f036ba03916&scope.domain.id=default&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 530 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:24.041821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:24.123579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb9e33-d6d2-48a9-9094-a161595cef66 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:24.160327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66ee6f-66e4-4bda-9291-3ee74f195df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:24.160965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ee6f-66e4-4bda-9291-3ee74f195df0 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:24.161250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ee6f-66e4-4bda-9291-3ee74f195df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:24.161524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ee6f-66e4-4bda-9291-3ee74f195df0 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:24.162196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66ee6f-66e4-4bda-9291-3ee74f195df0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:24.163694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66ee6f-66e4-4bda-9291-3ee74f195df0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:24.191026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 29/57] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Sep 9 22:48:24 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:48:26.334417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a3544-de93-4660-ae61-6bb353d3eeaa None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:26.334941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a3544-de93-4660-ae61-6bb353d3eeaa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:26.334941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a3544-de93-4660-ae61-6bb353d3eeaa None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:26.336329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 29/58] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:26.341512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58901-fd6c-4b96-a730-dbd0fb1766a2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:26.341929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58901-fd6c-4b96-a730-dbd0fb1766a2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:26.342065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58901-fd6c-4b96-a730-dbd0fb1766a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:26.360610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b4c58901-fd6c-4b96-a730-dbd0fb1766a2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:26.361799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b4c58901-fd6c-4b96-a730-dbd0fb1766a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:26.426027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 30/59] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:26.693835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a3544-de93-4660-ae61-6bb353d3eeaa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:26.743216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:26.795622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a3544-de93-4660-ae61-6bb353d3eeaa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:26.826836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1444be4-3e81-4dd1-92f0-ccba6af50322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:26.827450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1444be4-3e81-4dd1-92f0-ccba6af50322 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:26.827758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1444be4-3e81-4dd1-92f0-ccba6af50322 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:26.828026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1444be4-3e81-4dd1-92f0-ccba6af50322 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:26.829114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1444be4-3e81-4dd1-92f0-ccba6af50322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:26.830885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1444be4-3e81-4dd1-92f0-ccba6af50322 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:26.832450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 30/60] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:26.837959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c58901-fd6c-4b96-a730-dbd0fb1766a2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:26.846291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:26.934577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c58901-fd6c-4b96-a730-dbd0fb1766a2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:26.967123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed860b0-03ba-4451-9e81-551a564988ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:26.967688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed860b0-03ba-4451-9e81-551a564988ad None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:26.967970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed860b0-03ba-4451-9e81-551a564988ad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:26.968244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed860b0-03ba-4451-9e81-551a564988ad None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:26.968888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed860b0-03ba-4451-9e81-551a564988ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:26.970345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed860b0-03ba-4451-9e81-551a564988ad None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:26.995506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 31/61] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:48:26 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:30.335985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cd910-2463-4e3d-9c9b-ec6303077529 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.336504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cd910-2463-4e3d-9c9b-ec6303077529 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.336585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cd910-2463-4e3d-9c9b-ec6303077529 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.337880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 31/62] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:30.343781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd479f-ea26-4dfd-82ab-619908023031 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.344575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd479f-ea26-4dfd-82ab-619908023031 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.344998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd479f-ea26-4dfd-82ab-619908023031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.368553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f8fd479f-ea26-4dfd-82ab-619908023031 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:30.370554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f8fd479f-ea26-4dfd-82ab-619908023031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:30.400637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ba013-03de-4d96-95f0-9c51eaf95379 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.401147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ba013-03de-4d96-95f0-9c51eaf95379 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.402103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ba013-03de-4d96-95f0-9c51eaf95379 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.403612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 32/63] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:30.411543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b209bc6-fe0f-41a1-96d6-0a3ef1ecebe1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.411543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b209bc6-fe0f-41a1-96d6-0a3ef1ecebe1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.411543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b209bc6-fe0f-41a1-96d6-0a3ef1ecebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.479889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:30.495862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 32/64] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:30.537396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1b209bc6-fe0f-41a1-96d6-0a3ef1ecebe1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:30.538987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1b209bc6-fe0f-41a1-96d6-0a3ef1ecebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:30.632949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 33/65] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:30.719896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbe29-8ca9-416e-bdcf-8579b0fa07e2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.720216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbe29-8ca9-416e-bdcf-8579b0fa07e2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.720467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbe29-8ca9-416e-bdcf-8579b0fa07e2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.721980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 33/66] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:30.723251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c14079-8c5d-4839-8b08-73a98371f957 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.723579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c14079-8c5d-4839-8b08-73a98371f957 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.723860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c14079-8c5d-4839-8b08-73a98371f957 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.725315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 34/67] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:30.736383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839504f-bdad-4ca9-a953-bf0f37554d86 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.736672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839504f-bdad-4ca9-a953-bf0f37554d86 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.737263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02af4c7-6010-4061-bd7a-1e7d650c111a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:30.737762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839504f-bdad-4ca9-a953-bf0f37554d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.739709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02af4c7-6010-4061-bd7a-1e7d650c111a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:30.740005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02af4c7-6010-4061-bd7a-1e7d650c111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:30.767017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0839504f-bdad-4ca9-a953-bf0f37554d86 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:30.768268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0839504f-bdad-4ca9-a953-bf0f37554d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:30.769379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e02af4c7-6010-4061-bd7a-1e7d650c111a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:30.771715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e02af4c7-6010-4061-bd7a-1e7d650c111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:30.839955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:30.866068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 34/68] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:30.866744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02af4c7-6010-4061-bd7a-1e7d650c111a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:30.874569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 35/69] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:30.973303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839504f-bdad-4ca9-a953-bf0f37554d86 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.011544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02af4c7-6010-4061-bd7a-1e7d650c111a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.036159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.048089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a6871-b1fd-4b27-9059-9391b04eeb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.048740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a6871-b1fd-4b27-9059-9391b04eeb48 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.049026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a6871-b1fd-4b27-9059-9391b04eeb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.049304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a6871-b1fd-4b27-9059-9391b04eeb48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.060475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a6871-b1fd-4b27-9059-9391b04eeb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-399a6871-b1fd-4b27-9059-9391b04eeb48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:31.065303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:31.066139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:31.066891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 35/70] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:31.068285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a846121-4073-4f37-8516-835e3d54ff26 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.068285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a846121-4073-4f37-8516-835e3d54ff26 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.073693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a846121-4073-4f37-8516-835e3d54ff26 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.073693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 36/71] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:31.075176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a846121-4073-4f37-8516-835e3d54ff26 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.118041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839504f-bdad-4ca9-a953-bf0f37554d86 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.160233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915c8fd-acb1-4e41-b4b0-a40cbd68c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.161227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915c8fd-acb1-4e41-b4b0-a40cbd68c68a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.161760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915c8fd-acb1-4e41-b4b0-a40cbd68c68a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.162218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915c8fd-acb1-4e41-b4b0-a40cbd68c68a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.173965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915c8fd-acb1-4e41-b4b0-a40cbd68c68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0915c8fd-acb1-4e41-b4b0-a40cbd68c68a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:31.179472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:31.180322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:31.181070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:31.184138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 36/72] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:48:31.186206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d0995-52d1-485e-8081-2ed87f8d6e7c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.186937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d0995-52d1-485e-8081-2ed87f8d6e7c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.187216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d0995-52d1-485e-8081-2ed87f8d6e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.198183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a846121-4073-4f37-8516-835e3d54ff26 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.205435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6f7d0995-52d1-485e-8081-2ed87f8d6e7c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:31.206641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6f7d0995-52d1-485e-8081-2ed87f8d6e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:31.242597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.242828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.243479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.243791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.244101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.244747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:31.245092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:31.246509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:31.254862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 37/73] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:31.255708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.285852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 37/74] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:31.286709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d0995-52d1-485e-8081-2ed87f8d6e7c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.302797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.358334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c240-f97d-4d45-ad65-9bde26a0d287 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.392787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.393424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.393705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.394140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.405279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:31.407285 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d0995-52d1-485e-8081-2ed87f8d6e7c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:31.410606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:31.411441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:31.412481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 38/75] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:31.417200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.445398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.446168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.446744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.447124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.447721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:31.448026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:31.450002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:31.461445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 38/76] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:31.461445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.487798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.525815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc67d27-83e9-4804-9bdb-36eaf4893d22 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.538189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.572252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac22f8-3b5c-4d34-ac14-0413116ca6ee None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.574541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.575185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.575471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.575780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.585712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:31.591751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:31.592709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:31.593475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 39/77] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:31.594136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.610139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.610774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.611056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.611327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.626892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:31.628386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:31.630033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 39/78] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:31.630033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.691759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4dd04-7501-4143-9a8e-04b7dbcf33a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.716543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.724764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.725375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.725654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.726123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.726597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:31.726890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:31.728303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:31.733978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd2cc3-8d89-4c1c-b2d2-153c182932a1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.738069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 40/79] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:31.738699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.766322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.769390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.770176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.770461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.771036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] PATH_INFO: `/v3/users/fa41a2a2137d4147a83f9f036ba03916` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.772344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41a2a2137d4147a83f9f036ba03916)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:31.773774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:31.780607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 40/80] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/users/fa41a2a2137d4147a83f9f036ba03916 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:31.780607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.839160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6f945-962e-43da-a6ae-474baef8d991 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.871968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.872595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.872874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.873147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.873234 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c5bc-25d2-49d4-8424-295816ca8f17 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:31.874259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:31.875756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:31.877313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 41/81] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:31.878078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.906486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:31.907091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:31.907370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:31.907679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:31.908338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:31.908638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:31.910066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:31.918390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 41/82] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:31.918974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:31.936854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.971831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:31.978455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bd681-76d4-4632-a6f9-09264acfba7b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.012021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.012629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.012911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.013187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.016135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69255-0f7c-44ea-ab9d-a330cdd85481 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.048209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.048764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.049043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.049312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.050432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.051907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.053464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 42/83] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:32.054300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.079117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.080630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.136430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.152167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef75390-e897-4a3a-b585-37b6748d72a2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.170390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.178000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 42/84] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:32.178788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.189785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.190406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.190692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.190969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.256505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.257974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.274243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefce60-9635-48ca-9999-b97966d69294 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.308504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.309034 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.309314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.309588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.310238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.311738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.332283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 43/85] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:48:32 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:32.333099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.353086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.374972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 43/86] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:32.379863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.392244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.454187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf7247-3c53-4228-870a-9fb0d6034c6c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.490936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7979d11-7018-4d42-935e-6d5e9527d956 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.491750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.492533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.492820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.493099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.494426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.496013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.503101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 44/87] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:32 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:32.503101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.537663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.538280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.538562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.538837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.601271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.620636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.622414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.631757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b761781-5d62-4d15-8d6f-f53d0e002255 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.660780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.679689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.680624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.681417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.681871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.683612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.685269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.687036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 45/88] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:32 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:32.687831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.752103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 44/89] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:32.753205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.819472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b7ef7-46ae-4d25-9973-28a33bbe9d6c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.842346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.861275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.861477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddb789-8c46-463d-a9c5-a72c4adc4e39 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:32.862249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.862911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.863329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.865547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.867185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:32.869736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 46/90] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:32 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:32.871443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:32.910000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:32.910879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:32.911174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:32.911459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:32.944130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:32.997698 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:32.999315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:33.003895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96145ff-824d-41e9-b0c9-79cf18a9a6e8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:33.049610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:33.050778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:33.051475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:33.051922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:33.052982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=8db4ab0dc8ec4234b9f8664feb8161e9) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:33.053779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:33.055739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:33.064869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 47/91] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:32 2022] GET /identity/v3/role_assignments?user.id=8db4ab0dc8ec4234b9f8664feb8161e9&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:33.066166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:33.095882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:33.122162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 45/92] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:33.122162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:33.232470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f7f3b-2b1a-4c5a-b573-ebcf619fbf2a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:33.275886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:33.302674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cb844-0b0a-4c80-bf43-9aac77b4e8fb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:33.308614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:33.309580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:33.310068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:33.310539 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] PATH_INFO: `/v3/projects/428276fb5de248aba5212002b5a59fbb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:33.312648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] RBAC: Authorizing `identity:get_project(project_id=428276fb5de248aba5212002b5a59fbb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:33.314749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:33.317125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 48/93] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:33 2022] GET /identity/v3/projects/428276fb5de248aba5212002b5a59fbb => generated 312 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:33.318597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:33.348440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:33.349060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:33.349354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:33.349655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:33.350966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=fa41a2a2137d4147a83f9f036ba03916) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:33.351289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:33.353172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:33.361449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 46/94] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:33 2022] GET /identity/v3/role_assignments?user.id=fa41a2a2137d4147a83f9f036ba03916&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:33.362614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:33.496143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:33.529153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db92c93-e90d-4a32-92fb-c747d825fb91 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:33.549597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f66e7c-e8eb-4808-9d48-50dd338ec0a6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:33.568502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:33.569475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:33.569979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:33.570445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:33.571368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=428276fb5de248aba5212002b5a59fbb, user.id=8db4ab0dc8ec4234b9f8664feb8161e9) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:33.572034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:33.573753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:33.579753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 47/95] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:33 2022] GET /identity/v3/role_assignments?user.id=8db4ab0dc8ec4234b9f8664feb8161e9&scope.project.id=428276fb5de248aba5212002b5a59fbb&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:33.636538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f8b32-f793-4ac5-b0b6-5f540fa739c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:33.637225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f8b32-f793-4ac5-b0b6-5f540fa739c3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:33.637511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f8b32-f793-4ac5-b0b6-5f540fa739c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:33.637782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f8b32-f793-4ac5-b0b6-5f540fa739c3 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:33.639088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f8b32-f793-4ac5-b0b6-5f540fa739c3 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:33.640603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f8b32-f793-4ac5-b0b6-5f540fa739c3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:33.642293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:33 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:33.648614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:33.805351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e592c45-3109-402b-a5c3-04c3bed4a5b5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:33.824609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:33.852661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55191fff-8172-483a-8c56-05560b367b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:33.853296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191fff-8172-483a-8c56-05560b367b7b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:33.853586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191fff-8172-483a-8c56-05560b367b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:33.853867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191fff-8172-483a-8c56-05560b367b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:33.855137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55191fff-8172-483a-8c56-05560b367b7b None admin] RBAC: Adding query filter params (role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=8db4ab0dc8ec4234b9f8664feb8161e9) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:33.855450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55191fff-8172-483a-8c56-05560b367b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:33.856926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55191fff-8172-483a-8c56-05560b367b7b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:33.868173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:33 2022] GET /identity/v3/role_assignments?user.id=8db4ab0dc8ec4234b9f8664feb8161e9&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:35.338385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a736e6-a598-4cd5-a0f5-07b373568afe None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:35.339519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a736e6-a598-4cd5-a0f5-07b373568afe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:35.339616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a736e6-a598-4cd5-a0f5-07b373568afe None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:35.341204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:35.346495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393beccb-c6e1-4215-93b0-f9b33346d1f1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:35.346891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393beccb-c6e1-4215-93b0-f9b33346d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:35.347052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393beccb-c6e1-4215-93b0-f9b33346d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:35.365334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-393beccb-c6e1-4215-93b0-f9b33346d1f1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:35.366530 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-393beccb-c6e1-4215-93b0-f9b33346d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:35.411080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bfc4b-86f0-4598-88e3-6f25573c399d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:35.411377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bfc4b-86f0-4598-88e3-6f25573c399d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:35.411659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bfc4b-86f0-4598-88e3-6f25573c399d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:35.412994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 51/99] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:35.428853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c99b1b-8588-4241-b488-bb26bbf915f0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:35.429196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c99b1b-8588-4241-b488-bb26bbf915f0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:35.429480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c99b1b-8588-4241-b488-bb26bbf915f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:35.454632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-35c99b1b-8588-4241-b488-bb26bbf915f0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:35.455879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35c99b1b-8588-4241-b488-bb26bbf915f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:35.463206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 49/100] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:35.512388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:35.537043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 52/101] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:35.864892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393beccb-c6e1-4215-93b0-f9b33346d1f1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:35.984562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393beccb-c6e1-4215-93b0-f9b33346d1f1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:35.997720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:36.035405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:36.036109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:36.036391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:36.036665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:36.037895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:36.039382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:36.041255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 50/102] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:36.047778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c99b1b-8588-4241-b488-bb26bbf915f0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:36.060118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:36.195224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c99b1b-8588-4241-b488-bb26bbf915f0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:36.230992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5f3ea-54b3-485d-b904-9172db821419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:36.232271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5f3ea-54b3-485d-b904-9172db821419 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:36.232749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5f3ea-54b3-485d-b904-9172db821419 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:36.233232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5f3ea-54b3-485d-b904-9172db821419 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:36.234172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5f3ea-54b3-485d-b904-9172db821419 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:36.235969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5f3ea-54b3-485d-b904-9172db821419 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:36.265984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a7b1e-07fd-4825-a35f-a397efc4f42c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:36.267507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 53/103] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:48:36 2022] POST /identity/v3/projects => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:36.270004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3898f6-027c-4699-85ab-ab521b2d1c57 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:36.270296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3898f6-027c-4699-85ab-ab521b2d1c57 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:36.270568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3898f6-027c-4699-85ab-ab521b2d1c57 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:36.276924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 54/104] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:36.281980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02aebf-d2c7-46c5-8fee-b20818f39150 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:36.282587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02aebf-d2c7-46c5-8fee-b20818f39150 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:36.283097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02aebf-d2c7-46c5-8fee-b20818f39150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:36.315431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:36.316475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:36.316965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:36.317440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:36.319370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9c02aebf-d2c7-46c5-8fee-b20818f39150 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:36.320627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9c02aebf-d2c7-46c5-8fee-b20818f39150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:36.338321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:36.347028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:36.348423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:36.349219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:36.358437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 51/105] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:36.364119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:36.397837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:36.445267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:36.486875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 55/106] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:36.537257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427c45c-c1b2-43d6-a725-94535ff16c71 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:36.575936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:36.576963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:36.577450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:36.577949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:36.578809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:36.579311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:36.580931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:36.590819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 52/107] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:36.600947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02aebf-d2c7-46c5-8fee-b20818f39150 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:36.734738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02aebf-d2c7-46c5-8fee-b20818f39150 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:36.779470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:36.780172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:36.780461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:36.780735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] PATH_INFO: `/v3/users/a400986567484eca8450e4ab365adf8f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:36.809305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] RBAC: Authorizing `identity:get_user(user_id=a400986567484eca8450e4ab365adf8f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:36.810834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:36.812983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 56/108] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:36 2022] GET /identity/v3/users/a400986567484eca8450e4ab365adf8f => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:36.818580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:36.915896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:36.956824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:36.971854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:36.972236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2438cc-2a52-4dab-8385-b8fe7f198788 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:37.041406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:37.042510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.042791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.043064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.113666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd4485-58c8-42a8-89cd-317cde9c672b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:37.138100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:37.140132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:37.151837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee80993-9488-4950-bf68-ec77d8f0a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:37.152641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80993-9488-4950-bf68-ec77d8f0a210 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.152925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80993-9488-4950-bf68-ec77d8f0a210 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.153196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80993-9488-4950-bf68-ec77d8f0a210 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.164711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee80993-9488-4950-bf68-ec77d8f0a210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2ee80993-9488-4950-bf68-ec77d8f0a210 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:37.170581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:37.171433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:37.172245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 57/109] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:37.173607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5e7f0-9d79-43cd-a866-97c425c37b98 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.174728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5e7f0-9d79-43cd-a866-97c425c37b98 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.174728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5e7f0-9d79-43cd-a866-97c425c37b98 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.175624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 58/110] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:37.176850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628a6a-0a2c-4222-be61-cf79bdcbd6d4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.177275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628a6a-0a2c-4222-be61-cf79bdcbd6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.177549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628a6a-0a2c-4222-be61-cf79bdcbd6d4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.178724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 59/111] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:37.184092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9d532-06d9-49b9-b0c1-42cd1b740816 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.184496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9d532-06d9-49b9-b0c1-42cd1b740816 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.184810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9d532-06d9-49b9-b0c1-42cd1b740816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.212524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-44c9d532-06d9-49b9-b0c1-42cd1b740816 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:37.214554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-44c9d532-06d9-49b9-b0c1-42cd1b740816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:37.247895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:37.257729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 53/112] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:37.258574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:37.310592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 60/113] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:37.311783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a34b9b-b453-47e7-9fdf-8e40cb4c07d3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.312080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a34b9b-b453-47e7-9fdf-8e40cb4c07d3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.312362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a34b9b-b453-47e7-9fdf-8e40cb4c07d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.330326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-61a34b9b-b453-47e7-9fdf-8e40cb4c07d3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:37.331559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-61a34b9b-b453-47e7-9fdf-8e40cb4c07d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:37.342365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:37.392848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b9652-9763-4077-bac8-9952023f9bb4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:37.439423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:37.443173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.443342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.443558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.445500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:37.445815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:37.447564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:37.464924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 54/114] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:37.469448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 61/115] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:37.470582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:37.471623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a34b9b-b453-47e7-9fdf-8e40cb4c07d3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:37.579945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:37.660948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a34b9b-b453-47e7-9fdf-8e40cb4c07d3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:37.663450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c68ee9-2963-46fe-89be-8b87a8baaa8a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:37.711250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473e62-9c00-440d-b049-24cdaf919725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:37.712004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473e62-9c00-440d-b049-24cdaf919725 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.712298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473e62-9c00-440d-b049-24cdaf919725 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.712580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473e62-9c00-440d-b049-24cdaf919725 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.713797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473e62-9c00-440d-b049-24cdaf919725 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:37.715283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473e62-9c00-440d-b049-24cdaf919725 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:37.717256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 55/116] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:37 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:37.718424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c6358-7978-445f-bfc2-8a96f5285be6 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.718714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c6358-7978-445f-bfc2-8a96f5285be6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.718994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c6358-7978-445f-bfc2-8a96f5285be6 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.720469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 56/117] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:37.721187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c6358-7978-445f-bfc2-8a96f5285be6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:37.755306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:37.755976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.756258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.756536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.758317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:37.759802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:37.762139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 62/118] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:37 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:37.762679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:37.778126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:37.845062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c6358-7978-445f-bfc2-8a96f5285be6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:37.871437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:37.871885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f6a9-ea28-4542-b339-39c762c2830a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:37.881676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203519f2-ff40-4930-8af3-aa7e9ff92a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:37.882307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203519f2-ff40-4930-8af3-aa7e9ff92a1d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.882777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203519f2-ff40-4930-8af3-aa7e9ff92a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.882882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203519f2-ff40-4930-8af3-aa7e9ff92a1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.895373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203519f2-ff40-4930-8af3-aa7e9ff92a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-203519f2-ff40-4930-8af3-aa7e9ff92a1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:37.900367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:37.901255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:37.902030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 57/119] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:37.902671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436f05-d37a-4f9a-94b2-f79724bc814e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.907707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436f05-d37a-4f9a-94b2-f79724bc814e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.907707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436f05-d37a-4f9a-94b2-f79724bc814e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.913560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:37.914172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:37.914481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:37.914779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:37.915533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] RBAC: Adding query filter params (role.id=78d8f7f202354798acd7876e3bf7deff, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=a400986567484eca8450e4ab365adf8f) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:37.915846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:37.917326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:37.923026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 63/120] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:37 2022] GET /identity/v3/role_assignments?user.id=a400986567484eca8450e4ab365adf8f&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=78d8f7f202354798acd7876e3bf7deff => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:37.923800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fd436f05-d37a-4f9a-94b2-f79724bc814e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:37.923909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:37.925268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fd436f05-d37a-4f9a-94b2-f79724bc814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:38.008358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 58/121] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:38.009614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436f05-d37a-4f9a-94b2-f79724bc814e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.051262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:38.096242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31d25d-6e3b-46a7-ad25-8a899722eca4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.151504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.152168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.154227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.154518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.156648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:38.158242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:38.160061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 64/122] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:37 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:38.160909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.174174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436f05-d37a-4f9a-94b2-f79724bc814e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.192013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:38.211935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.212586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.212876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.213156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.223889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:38.230486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:38.233200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 59/123] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:38.236968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.348421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece1d44-2e9d-4b41-bcea-f07f1b8a8c55 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.386381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.387039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.387322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.387622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.388439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:38.388843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:38.390285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:38.391748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14245de8-0c72-46fd-a88a-97901ab38ddd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.399397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 65/124] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:38.400480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.410501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:38.428565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.429333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.429630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.429911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/8db4ab0dc8ec4234b9f8664feb8161e9/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.431924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] RBAC: Authorizing `identity:create_grant(project_id=cda287745a6b42a3945fe2fed58f8120, user_id=8db4ab0dc8ec4234b9f8664feb8161e9, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:38.433619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:38.450784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 60/125] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:38 2022] PUT /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/8db4ab0dc8ec4234b9f8664feb8161e9/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:48:38.451454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.529055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:38.584544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4e0ba-fbc8-4ad9-9451-174b403ac574 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.657013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.657560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.657841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.658321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.658864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:38.659170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:38.661667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:38.666415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84940df7-cebb-44f2-9bf8-8135ba714bff None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.673737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 66/126] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:38.677474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.710499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.713691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.713691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.713691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.732163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:38.736293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:38.737142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 61/127] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:38.738725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.778557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:38.859474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934e7c-e634-4e60-93dc-30ddeb22abf0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.867974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:38.869614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d563e-5af6-4591-9949-533a9ecff141 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:38.899709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.900359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.900803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.901622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] PATH_INFO: `/v3/users/fa41a2a2137d4147a83f9f036ba03916` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.902785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41a2a2137d4147a83f9f036ba03916)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:38.904452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:38.906293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 62/128] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/users/fa41a2a2137d4147a83f9f036ba03916 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:38.906994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:38.958030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:38.958672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:38.958958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:38.959295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:38.965495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:38.967077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:38.973929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 67/129] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:38.975658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:39.033788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe00769-4e23-435c-ae89-a73ccc55f902 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:39.073500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:39.074236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.074809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.075484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:39.075826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:39.077522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:39.086739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 63/130] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:39.088154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:39.163876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:39.164190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d28115-a9e3-4700-9f09-4b4173ba8186 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:39.200552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:39.201526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.202028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.202492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.203853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:39.205604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:39.207748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 68/131] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:38 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:39.208780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:39.214829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:39.225826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd0dc3-a764-43f2-be25-335afbed8df8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:39.268503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8d19-03b9-4b54-87b0-e15a781250df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:39.269178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8d19-03b9-4b54-87b0-e15a781250df None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.269681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8d19-03b9-4b54-87b0-e15a781250df None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.269964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8d19-03b9-4b54-87b0-e15a781250df None admin] PATH_INFO: `/v3/projects/428276fb5de248aba5212002b5a59fbb` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.271212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8d19-03b9-4b54-87b0-e15a781250df None admin] RBAC: Authorizing `identity:get_project(project_id=428276fb5de248aba5212002b5a59fbb)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:39.272768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8d19-03b9-4b54-87b0-e15a781250df None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:39.281912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 64/132] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:39 2022] GET /identity/v3/projects/428276fb5de248aba5212002b5a59fbb => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:39.281912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef7320-7ebb-4bac-911e-c8bc8c8f684c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.281912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef7320-7ebb-4bac-911e-c8bc8c8f684c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.281912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef7320-7ebb-4bac-911e-c8bc8c8f684c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.281912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 65/133] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:39.281912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef7320-7ebb-4bac-911e-c8bc8c8f684c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:39.331745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c37bc8-5862-4ba0-a382-ecb39200f489 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:39.373922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee2fe2-46b9-48c6-b036-6103d2a2ac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:39.374603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee2fe2-46b9-48c6-b036-6103d2a2ac8b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.374889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee2fe2-46b9-48c6-b036-6103d2a2ac8b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.375161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee2fe2-46b9-48c6-b036-6103d2a2ac8b None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.376831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee2fe2-46b9-48c6-b036-6103d2a2ac8b None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:39.378970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee2fe2-46b9-48c6-b036-6103d2a2ac8b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:39.385310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 69/134] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:39 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:39.385310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbe29e-35ef-4df2-b074-6aecced36308 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.385310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbe29e-35ef-4df2-b074-6aecced36308 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.385310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbe29e-35ef-4df2-b074-6aecced36308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.402107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:39.409633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-78dbe29e-35ef-4df2-b074-6aecced36308 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:39.411227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78dbe29e-35ef-4df2-b074-6aecced36308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:39.412168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef7320-7ebb-4bac-911e-c8bc8c8f684c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:39.452659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:39.454714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.455191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.455770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/fa41a2a2137d4147a83f9f036ba03916/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.458188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] RBAC: Authorizing `identity:create_grant(project_id=cda287745a6b42a3945fe2fed58f8120, user_id=fa41a2a2137d4147a83f9f036ba03916, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:39.460133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:39.478948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 66/135] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:39 2022] PUT /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/fa41a2a2137d4147a83f9f036ba03916/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:48:39.480230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:39.528170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 70/136] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:39.531117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbe29e-35ef-4df2-b074-6aecced36308 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:39.537704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:39.744744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbe29e-35ef-4df2-b074-6aecced36308 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:39.748736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe1158-c1b8-4b2d-908f-3063f3c7263f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:39.809122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:39.809853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.810097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.810351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] PATH_INFO: `/v3/projects/428276fb5de248aba5212002b5a59fbb/users/8db4ab0dc8ec4234b9f8664feb8161e9/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.812937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] RBAC: Authorizing `identity:create_grant(project_id=428276fb5de248aba5212002b5a59fbb, user_id=8db4ab0dc8ec4234b9f8664feb8161e9, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:39.814393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:39.817559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:39.818205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:39.818485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:39.818756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:39.819919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:39.821350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:39.825091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 71/137] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:39 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:39.830891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:39.849324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 67/138] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:39 2022] PUT /identity/v3/projects/428276fb5de248aba5212002b5a59fbb/users/8db4ab0dc8ec4234b9f8664feb8161e9/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:48:39.925093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:39.971903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57046b0-380e-442c-b6b1-774c61e410a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:40.013821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:40.017622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:40.018300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:40.018594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:40.018865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/8db4ab0dc8ec4234b9f8664feb8161e9/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:40.032703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] RBAC: Authorizing `identity:create_grant(project_id=cda287745a6b42a3945fe2fed58f8120, user_id=8db4ab0dc8ec4234b9f8664feb8161e9, role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:40.034602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:40.045674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:40.067510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 72/139] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:39 2022] PUT /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/8db4ab0dc8ec4234b9f8664feb8161e9/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 22:48:40.183785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849cf75-51d0-424d-b07e-c137414c3668 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:40.220174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217fb45-d520-43a1-96fd-1968cdce774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:40.220825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217fb45-d520-43a1-96fd-1968cdce774c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:40.221111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217fb45-d520-43a1-96fd-1968cdce774c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:40.221448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217fb45-d520-43a1-96fd-1968cdce774c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:40.222633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217fb45-d520-43a1-96fd-1968cdce774c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:40.228089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217fb45-d520-43a1-96fd-1968cdce774c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:40.230250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 68/140] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:40.236233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:40.381797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f79d09-e8ce-432d-b517-bf805a2c2777 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:40.414874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:40.423185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a39f6-bbbc-48f2-9cfc-2978842548d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:40.424128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a39f6-bbbc-48f2-9cfc-2978842548d6 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:40.424708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a39f6-bbbc-48f2-9cfc-2978842548d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:40.425185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a39f6-bbbc-48f2-9cfc-2978842548d6 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:40.425874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a39f6-bbbc-48f2-9cfc-2978842548d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:40.427348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a39f6-bbbc-48f2-9cfc-2978842548d6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:40.462860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 73/141] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:48:40 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:40.472075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681df25-91a8-4a7d-bc5c-400f06628149 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:40.472587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681df25-91a8-4a7d-bc5c-400f06628149 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:40.472789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681df25-91a8-4a7d-bc5c-400f06628149 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:40.474414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 69/142] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:40.480191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da21205-5717-4ddd-9422-505459c85933 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:40.480483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da21205-5717-4ddd-9422-505459c85933 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:40.480760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da21205-5717-4ddd-9422-505459c85933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:40.501187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9da21205-5717-4ddd-9422-505459c85933 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:40.502469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9da21205-5717-4ddd-9422-505459c85933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:40.586621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 74/143] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:40.884778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681df25-91a8-4a7d-bc5c-400f06628149 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:41.004109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:41.020145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681df25-91a8-4a7d-bc5c-400f06628149 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:41.064531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:41.065175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:41.065464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:41.065741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:41.081176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:41.086079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:41.086916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:41.087735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 70/144] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:41.100612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da21205-5717-4ddd-9422-505459c85933 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:41.240015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da21205-5717-4ddd-9422-505459c85933 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:41.259780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:41.283776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:41.284395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:41.284674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:41.284942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:41.285642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:41.285911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:41.287303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:41.299891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 75/145] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:41.305619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:41.453352 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4017fb2-9433-44e5-8991-e2144020a13b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:41.498152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5e287-e753-425a-9bd7-1788c908a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:41.498902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5e287-e753-425a-9bd7-1788c908a926 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:41.499114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5e287-e753-425a-9bd7-1788c908a926 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:41.499393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5e287-e753-425a-9bd7-1788c908a926 None admin] PATH_INFO: `/v3/users/a400986567484eca8450e4ab365adf8f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:41.500562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5e287-e753-425a-9bd7-1788c908a926 None admin] RBAC: Authorizing `identity:get_user(user_id=a400986567484eca8450e4ab365adf8f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:41.502001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5e287-e753-425a-9bd7-1788c908a926 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:41.504577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 71/146] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:41 2022] GET /identity/v3/users/a400986567484eca8450e4ab365adf8f => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:41.510116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:41.632896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18042d61-e036-47f9-923a-9de164f9b16f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:41.646267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59126ba-b258-4ddd-87a4-04a384ee2729 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:41.646587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59126ba-b258-4ddd-87a4-04a384ee2729 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:41.646818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59126ba-b258-4ddd-87a4-04a384ee2729 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:41.648194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 72/147] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:41.654197 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531eda5-cec1-41b5-abed-9d733a053fca None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:41.654478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531eda5-cec1-41b5-abed-9d733a053fca None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:41.654749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531eda5-cec1-41b5-abed-9d733a053fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:41.676420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2531eda5-cec1-41b5-abed-9d733a053fca None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:41.678835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2531eda5-cec1-41b5-abed-9d733a053fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:41.683716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:41.689375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:41.690831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:41.691268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:41.691727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:41.693406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:41.695673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:41.697641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 76/148] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:41 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:41.704060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:41.769048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 73/149] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:41.778984 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:41.848276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224549c7-aa07-45b6-931f-57d278ad819e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:41.888195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:41.889259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:41.889737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:41.890209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/a400986567484eca8450e4ab365adf8f/roles/78d8f7f202354798acd7876e3bf7deff` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:41.896216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] RBAC: Authorizing `identity:create_grant(project_id=cda287745a6b42a3945fe2fed58f8120, user_id=a400986567484eca8450e4ab365adf8f, role_id=78d8f7f202354798acd7876e3bf7deff)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:41.897893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:41.922010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 77/150] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:41 2022] PUT /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120/users/a400986567484eca8450e4ab365adf8f/roles/78d8f7f202354798acd7876e3bf7deff => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 22:48:42.170684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531eda5-cec1-41b5-abed-9d733a053fca None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:42.295520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531eda5-cec1-41b5-abed-9d733a053fca None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:42.330549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:42.331555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:42.332045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:42.332510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:42.345187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:42.351671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:42.353125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:42.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:42.354907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:42.357888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 74/151] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:42.363734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:42.480401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f546f-9d73-4ff3-bfcf-960f2167a99b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:42.489505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:42.522915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:42.523547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:42.523827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:42.524093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:42.524763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:42.525065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:42.526966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:42.539595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 78/152] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:42.545829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:42.652321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a1e66-c452-4b25-8cc2-4a097822f040 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:42.689481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:42.690119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:42.690396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:42.690666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:42.691843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:42.693414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:42.695129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 75/153] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:42 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:42.701387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:42.837827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522479a-bfdd-4134-bc02-2c16998e42b7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:42.881639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:42.882272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:42.882550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:42.882815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:42.919583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:42.955987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:42.957464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:43.063946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 79/154] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:43.069837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:43.139333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:43.177986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1e40a-fa49-4919-ae5b-c008db422893 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:43.212958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6976dda-d3fb-42d9-b8e1-6bb89bfd5776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:43.214181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976dda-d3fb-42d9-b8e1-6bb89bfd5776 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.214467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976dda-d3fb-42d9-b8e1-6bb89bfd5776 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.214729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976dda-d3fb-42d9-b8e1-6bb89bfd5776 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.215887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6976dda-d3fb-42d9-b8e1-6bb89bfd5776 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:43.217321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6976dda-d3fb-42d9-b8e1-6bb89bfd5776 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:43.219287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 76/155] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:43 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:43.225033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:43.302252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:43.342708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f476-f89e-4f44-a943-0c369d0ef6aa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:43.379897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ed3b8-bd9d-4897-984f-fb4b361ec2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:43.380626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ed3b8-bd9d-4897-984f-fb4b361ec2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.380796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ed3b8-bd9d-4897-984f-fb4b361ec2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.381073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ed3b8-bd9d-4897-984f-fb4b361ec2d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.381799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ed3b8-bd9d-4897-984f-fb4b361ec2d8 None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=8db4ab0dc8ec4234b9f8664feb8161e9) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:43.382104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ed3b8-bd9d-4897-984f-fb4b361ec2d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:43.383578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ed3b8-bd9d-4897-984f-fb4b361ec2d8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:43.389807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 80/156] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:43 2022] GET /identity/v3/role_assignments?user.id=8db4ab0dc8ec4234b9f8664feb8161e9&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:43.755777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6c112-b3a1-4d6b-9374-edbc6cd7998b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.756451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6c112-b3a1-4d6b-9374-edbc6cd7998b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.756451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6c112-b3a1-4d6b-9374-edbc6cd7998b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.758041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 77/157] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:43.763965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af45de-363b-4693-a4d0-3a6de0fb9fc3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.764238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af45de-363b-4693-a4d0-3a6de0fb9fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.764512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af45de-363b-4693-a4d0-3a6de0fb9fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.791802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-05af45de-363b-4693-a4d0-3a6de0fb9fc3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:43.793002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05af45de-363b-4693-a4d0-3a6de0fb9fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:43.880862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc076d-7935-46b7-b32d-c323ed8e4b6e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.881180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc076d-7935-46b7-b32d-c323ed8e4b6e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.881466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc076d-7935-46b7-b32d-c323ed8e4b6e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.882793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 78/158] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:43.891126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74611d-679f-4608-ad7c-faee07052995 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.891391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74611d-679f-4608-ad7c-faee07052995 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.891705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74611d-679f-4608-ad7c-faee07052995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.915764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 81/159] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:43.919759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6a74611d-679f-4608-ad7c-faee07052995 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:43.921205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6a74611d-679f-4608-ad7c-faee07052995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:43.931970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189786c3-3e58-4bd7-a4f2-a280a59e9838 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.932888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189786c3-3e58-4bd7-a4f2-a280a59e9838 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.933054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189786c3-3e58-4bd7-a4f2-a280a59e9838 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.934476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 82/160] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:43.946460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5772e3-ad3a-4205-b63b-6727f00a7da8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:43.946733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5772e3-ad3a-4205-b63b-6727f00a7da8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:43.946993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5772e3-ad3a-4205-b63b-6727f00a7da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:43.977057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0d5772e3-ad3a-4205-b63b-6727f00a7da8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:43.978263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0d5772e3-ad3a-4205-b63b-6727f00a7da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:44.051122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 79/161] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:44.075319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:44.087364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 83/162] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:44.462316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74611d-679f-4608-ad7c-faee07052995 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:44.476456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5772e3-ad3a-4205-b63b-6727f00a7da8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:44.583132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:44.596247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5772e3-ad3a-4205-b63b-6727f00a7da8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:44.645914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:44.647172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:44.647655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:44.648234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:44.659250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74611d-679f-4608-ad7c-faee07052995 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:44.670609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:44.676293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:44.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:44.677950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:44.685903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 84/163] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:44.686882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:44.780191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:44.787892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:44.787892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:44.787892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:44.787892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:44.816171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:44.821226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:44.822132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:44.823461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:44.828180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 80/164] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:48:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:44.829136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:44.862781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa9c0e-647a-4727-b1a6-9def83c24776 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:44.926955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:44.927642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:44.927927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:44.928204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:44.948944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:44.953903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:44.954765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:44.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 85/165] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:44.956245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:45.073681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c870d-9a43-459c-829d-48800640c681 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:45.137359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:45.152109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee02f-c73e-4d21-8f4c-38f1f29d9fe9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:45.160484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:45.161151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.161434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.161694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.162355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:45.162649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:45.164067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:45.169469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:45.185061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 81/166] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:45.186128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:45.211102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b430a31-422f-4601-82d0-87c654a3d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:45.211743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b430a31-422f-4601-82d0-87c654a3d037 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.212022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b430a31-422f-4601-82d0-87c654a3d037 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.212297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b430a31-422f-4601-82d0-87c654a3d037 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.212970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b430a31-422f-4601-82d0-87c654a3d037 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:45.213331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b430a31-422f-4601-82d0-87c654a3d037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:45.214742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b430a31-422f-4601-82d0-87c654a3d037 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:45.233813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 86/167] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:48:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:45.235930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079534b-e1fc-4a1c-a66a-14b84416bcde None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.235930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079534b-e1fc-4a1c-a66a-14b84416bcde None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.236427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079534b-e1fc-4a1c-a66a-14b84416bcde None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.238014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 87/168] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:45.238966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079534b-e1fc-4a1c-a66a-14b84416bcde None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:45.341464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8fe0-fb84-48f5-9ce7-d2c5052029cd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:45.385641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:45.386275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.386557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.386829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.387526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:45.387831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:45.395511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:45.398225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 82/169] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:45.398889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:45.425968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079534b-e1fc-4a1c-a66a-14b84416bcde None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:45.499065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:45.500384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d3056-5709-4757-8d86-f4509f328dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:45.501018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d3056-5709-4757-8d86-f4509f328dae None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.501295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d3056-5709-4757-8d86-f4509f328dae None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.501574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d3056-5709-4757-8d86-f4509f328dae None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.506531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d3056-5709-4757-8d86-f4509f328dae None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:45.508027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d3056-5709-4757-8d86-f4509f328dae None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:45.510151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 88/170] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:45 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:45.511469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0523076-3402-46ad-b67f-1e1359c5946a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.511725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0523076-3402-46ad-b67f-1e1359c5946a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.511972 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0523076-3402-46ad-b67f-1e1359c5946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.549263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d0523076-3402-46ad-b67f-1e1359c5946a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:45.550495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d0523076-3402-46ad-b67f-1e1359c5946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:45.568447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134d043-e49e-462f-ae15-1a1e8d87bac7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:45.613677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:45.614770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.620491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.620812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] PATH_INFO: `/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.622113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] RBAC: Authorizing `identity:get_user(user_id=8db4ab0dc8ec4234b9f8664feb8161e9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:45.623668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:45.624614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:45.630022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 83/171] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:45 2022] GET /identity/v3/users/8db4ab0dc8ec4234b9f8664feb8161e9 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:45.631011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:45.666742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 89/172] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:45.669434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0523076-3402-46ad-b67f-1e1359c5946a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:45.773237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f90a77-8855-4b2b-a47b-18c364848caa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:45.794613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:45.827259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:45.827259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:45.827259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:45.827259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] PATH_INFO: `/v3/users/fa41a2a2137d4147a83f9f036ba03916` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:45.827744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41a2a2137d4147a83f9f036ba03916)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:45.827744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:45.827744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 84/173] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:45 2022] GET /identity/v3/users/fa41a2a2137d4147a83f9f036ba03916 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:45.827744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:45.924670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0523076-3402-46ad-b67f-1e1359c5946a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.005208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b38882-bb01-49b4-9178-00994082559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.006003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38882-bb01-49b4-9178-00994082559f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.006140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38882-bb01-49b4-9178-00994082559f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.006361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38882-bb01-49b4-9178-00994082559f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.026172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a950d3-886d-4876-82c1-dba5168282b1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.054676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.062097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.062707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.062981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.063248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.081922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b38882-bb01-49b4-9178-00994082559f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:46.083384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b38882-bb01-49b4-9178-00994082559f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:46.116702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.131387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:46.132883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:46.178859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 90/174] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:46.180084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7080316-04a1-4448-aa17-8359a38b45a4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.180482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7080316-04a1-4448-aa17-8359a38b45a4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.180674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7080316-04a1-4448-aa17-8359a38b45a4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.182543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 91/175] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:46.183116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7080316-04a1-4448-aa17-8359a38b45a4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.234685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 85/176] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:46.235773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.254499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.285010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7080316-04a1-4448-aa17-8359a38b45a4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.318589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf010a-8a32-4aa9-9a0a-41599041c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.319199 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf010a-8a32-4aa9-9a0a-41599041c765 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.319511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf010a-8a32-4aa9-9a0a-41599041c765 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.319790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf010a-8a32-4aa9-9a0a-41599041c765 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.320436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.336928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bef85-c81e-4322-aae3-a5ecb579ca7a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.371186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f67c8-6084-4161-9de1-e4770bedc48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.371838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f67c8-6084-4161-9de1-e4770bedc48a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.372122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f67c8-6084-4161-9de1-e4770bedc48a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.372396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f67c8-6084-4161-9de1-e4770bedc48a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.382305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f67c8-6084-4161-9de1-e4770bedc48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:46.384610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf010a-8a32-4aa9-9a0a-41599041c765 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:46.386845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf010a-8a32-4aa9-9a0a-41599041c765 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-dd9f67c8-6084-4161-9de1-e4770bedc48a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:46.387758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:46.388728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 86/177] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:46.392200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617108b-207b-4c8e-8900-3e84e37b30de None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.392916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617108b-207b-4c8e-8900-3e84e37b30de None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.392916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617108b-207b-4c8e-8900-3e84e37b30de None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.392916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 87/178] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:46.393091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3344-00ca-49c4-9694-86306bfef477 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.393309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3344-00ca-49c4-9694-86306bfef477 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.393581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3344-00ca-49c4-9694-86306bfef477 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.399258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 88/179] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:46.399258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe3344-00ca-49c4-9694-86306bfef477 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.462648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.493033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 92/180] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:46.494475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94631ff-e15a-43ff-a2fd-a8d64242de25 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.495318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94631ff-e15a-43ff-a2fd-a8d64242de25 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.495634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94631ff-e15a-43ff-a2fd-a8d64242de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.505209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe3344-00ca-49c4-9694-86306bfef477 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.513748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e94631ff-e15a-43ff-a2fd-a8d64242de25 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:46.515035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e94631ff-e15a-43ff-a2fd-a8d64242de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:46.537939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.538571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.538853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.539126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] PATH_INFO: `/v3/projects/428276fb5de248aba5212002b5a59fbb` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.540278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] RBAC: Authorizing `identity:get_project(project_id=428276fb5de248aba5212002b5a59fbb)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:46.542267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:46.543924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 89/181] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/projects/428276fb5de248aba5212002b5a59fbb => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:46.549682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.564224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.591575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 93/182] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:46.591575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94631ff-e15a-43ff-a2fd-a8d64242de25 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.650547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc1375-2901-4068-af64-7e85d76b921d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.682607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94631ff-e15a-43ff-a2fd-a8d64242de25 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.684150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.688042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4956449-5985-489e-882c-2193355bb58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.688681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956449-5985-489e-882c-2193355bb58a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.688962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956449-5985-489e-882c-2193355bb58a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.689229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956449-5985-489e-882c-2193355bb58a None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.690370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4956449-5985-489e-882c-2193355bb58a None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:46.691912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4956449-5985-489e-882c-2193355bb58a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:46.693744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 90/183] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:46.694701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdff744-352f-4d6f-b2a3-396a79b330dc None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.694990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdff744-352f-4d6f-b2a3-396a79b330dc None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.695260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdff744-352f-4d6f-b2a3-396a79b330dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.719047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1bdff744-352f-4d6f-b2a3-396a79b330dc None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:46.720322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1bdff744-352f-4d6f-b2a3-396a79b330dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:46.729588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49099ef4-9688-40bf-8c43-7fb52f773515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.730346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49099ef4-9688-40bf-8c43-7fb52f773515 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.730867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49099ef4-9688-40bf-8c43-7fb52f773515 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.731380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49099ef4-9688-40bf-8c43-7fb52f773515 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.732502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49099ef4-9688-40bf-8c43-7fb52f773515 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:46.733126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49099ef4-9688-40bf-8c43-7fb52f773515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:46.735188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49099ef4-9688-40bf-8c43-7fb52f773515 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:46.744441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 94/184] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:46.745659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a354d-f48a-4ec1-a123-c0a33163758a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.745947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a354d-f48a-4ec1-a123-c0a33163758a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.746221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a354d-f48a-4ec1-a123-c0a33163758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.765608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-432a354d-f48a-4ec1-a123-c0a33163758a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:46.767354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-432a354d-f48a-4ec1-a123-c0a33163758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:46.807534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 91/185] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:46.815920 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdff744-352f-4d6f-b2a3-396a79b330dc None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.824711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.859153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 95/186] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:46.860265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a354d-f48a-4ec1-a123-c0a33163758a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.928561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdff744-352f-4d6f-b2a3-396a79b330dc None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:46.962553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:46.963734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:46.964021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:46.964292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:46.965405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:46.966834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:46.968668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 92/187] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:46.968968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:46.969377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:46.979591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a354d-f48a-4ec1-a123-c0a33163758a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.025666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.027088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.028011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.028486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.029644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=428276fb5de248aba5212002b5a59fbb, user.id=8db4ab0dc8ec4234b9f8664feb8161e9) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:47.030395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.032795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.040749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 96/188] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/role_assignments?user.id=8db4ab0dc8ec4234b9f8664feb8161e9&scope.project.id=428276fb5de248aba5212002b5a59fbb&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:47.042389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.074676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:47.113828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3774b9-7875-429f-8c87-60947fa9ed8a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.164668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.165297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.165581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.165855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.167187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] RBAC: Adding query filter params (role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=8db4ab0dc8ec4234b9f8664feb8161e9) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:47.167515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.169401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.176905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 93/189] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:46 2022] GET /identity/v3/role_assignments?user.id=8db4ab0dc8ec4234b9f8664feb8161e9&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:47.177748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.185325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0adef-5d67-4e23-8d8c-9d89edbad621 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.232987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.233726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.234008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.234448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] PATH_INFO: `/v3/users/49d38627f8d04829871a044f09ddf75b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.252971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] RBAC: Authorizing `identity:get_user(user_id=49d38627f8d04829871a044f09ddf75b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.254452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.256781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 97/190] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/users/49d38627f8d04829871a044f09ddf75b => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:47.257492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.279628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:47.316251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3299d4-aa42-431b-94a0-306f9b24ef25 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.365746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.366535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.366785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.367026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.378607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:47.384524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30830cf1-8d77-43b5-8f8d-7c80aa5ea720 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:47.384668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:47.386009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:47.386801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 94/191] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:47.388140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.391155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:47.426917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.427624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.427827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.428093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.428810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=fa41a2a2137d4147a83f9f036ba03916) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:47.429112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.430542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.440090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 98/192] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/role_assignments?user.id=fa41a2a2137d4147a83f9f036ba03916&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:47.440090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.520060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b980bd-5a22-423b-be4c-537dcd466c58 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.538078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:47.561101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.562021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.562305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.562578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.563581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6009c13-4b10-44a9-a366-4ef50866fa51 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.563776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.565248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.566861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 95/193] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:47.568572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.600840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.601492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.601773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.602046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.614283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:47.619763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:47.620657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 99/194] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:47.622494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.656451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:47.691870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07fda-095a-422f-a5ae-2e24ac882a91 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.727125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.727682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.727973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.728247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.741599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcd426-582a-457d-87a4-b4480968d659 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.773441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.773988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.774266 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.774540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.775182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:47.775480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.777408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.785401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 100/195] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:47.786250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.788762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:47.796332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.797797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.862848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:47.885708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e28fc-5214-497d-bb28-bb3b607da984 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:47.894897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 96/196] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:47.895666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:47.920342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:47.921029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:47.921375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:47.921713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:47.922369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:47.923821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:47.945243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 101/197] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:48:47 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:47.952309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.003748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1708a9-08fd-4624-86ca-95aa75a3d2c1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.019658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:48.043468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.044034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.044307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.044593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.045526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:48.045831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.047223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.061858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 97/198] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:48.061858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.074952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e8738-b908-414a-bfad-b5860ce48f3e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.099204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:48.113125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.113923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.114200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.114469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] PATH_INFO: `/v3/users/a400986567484eca8450e4ab365adf8f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.115647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] RBAC: Authorizing `identity:get_user(user_id=a400986567484eca8450e4ab365adf8f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.117109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.118688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 102/199] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:47 2022] GET /identity/v3/users/a400986567484eca8450e4ab365adf8f => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:48.124313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.176301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8919d1-90db-47de-aefe-9201b66a8171 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.224289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.225058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.225415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.225765 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.226990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.228482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.230715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 98/200] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:48 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:48.231552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.252892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2605b-3cfc-44f0-883d-286309543e9b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.285052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:48.297385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.297556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.297882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.298050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] PATH_INFO: `/v3/users/49d38627f8d04829871a044f09ddf75b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.300502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] RBAC: Authorizing `identity:get_user(user_id=49d38627f8d04829871a044f09ddf75b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.301455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.303194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 103/201] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:48 2022] GET /identity/v3/users/49d38627f8d04829871a044f09ddf75b => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:48.303838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.351997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649b63-b522-4bcf-aafd-48f19ff00f45 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.353762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:48.392785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.393461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.393746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.394023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.436861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249d3f-edc7-4ec4-866e-16c511533de4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.472115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.473599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.481721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.482636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.483720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.484215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.485604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=49d38627f8d04829871a044f09ddf75b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:48.486121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.488495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.496152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 104/202] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:48 2022] GET /identity/v3/role_assignments?user.id=49d38627f8d04829871a044f09ddf75b&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:48.497791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.621353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:48.680228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 99/203] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:48.687154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.690898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:48.699358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108fd88-17ae-411d-b8e1-a216a5b65de9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.765874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.766730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.767150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.767581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.829722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15fdb3-fd42-46b2-958d-ac975b22c9fd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:48.893756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.895169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.908223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:48.908980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:48.909159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:48.909438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:48.910746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:48.912165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:48.914153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 100/204] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:48 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:48.920622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:48.978645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:49.042793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:49.059857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 105/205] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:49.066252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:49.090756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b32f5d-b72d-40b6-834c-f0c6c42080e7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:49.125702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:49.126345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:49.126632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:49.126912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:49.127702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] RBAC: Adding query filter params (role.id=78d8f7f202354798acd7876e3bf7deff, scope.project.id=cda287745a6b42a3945fe2fed58f8120, user.id=a400986567484eca8450e4ab365adf8f) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:49.128200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:49.129858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:49.137274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 101/206] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:48 2022] GET /identity/v3/role_assignments?user.id=a400986567484eca8450e4ab365adf8f&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=78d8f7f202354798acd7876e3bf7deff => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:49.262252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c093e9d-756b-47cd-a8d2-3e2abb2ef6b5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:49.311620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef57645-f468-4901-9aef-cbeb11fca29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:49.312478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef57645-f468-4901-9aef-cbeb11fca29b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:49.312769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef57645-f468-4901-9aef-cbeb11fca29b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:49.313052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef57645-f468-4901-9aef-cbeb11fca29b None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:49.314245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef57645-f468-4901-9aef-cbeb11fca29b None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:49.315785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef57645-f468-4901-9aef-cbeb11fca29b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:49.317907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 106/207] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:49 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:49.324638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:49.434634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862b063-8bf5-47ff-a334-f5a8a07c8394 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:49.470463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6886f-a815-41c1-b38a-b20e4d890b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:49.471076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6886f-a815-41c1-b38a-b20e4d890b80 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:49.471359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6886f-a815-41c1-b38a-b20e4d890b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:49.471664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6886f-a815-41c1-b38a-b20e4d890b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:49.472390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6886f-a815-41c1-b38a-b20e4d890b80 None admin] RBAC: Adding query filter params (role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=49d38627f8d04829871a044f09ddf75b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:49.472697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6886f-a815-41c1-b38a-b20e4d890b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:49.474152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6886f-a815-41c1-b38a-b20e4d890b80 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:49.476006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:49.481193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 102/208] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:49 2022] GET /identity/v3/role_assignments?user.id=49d38627f8d04829871a044f09ddf75b&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:50.429967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e3323-0896-4d79-9592-bc976dba06d3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:50.430516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e3323-0896-4d79-9592-bc976dba06d3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:50.430516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e3323-0896-4d79-9592-bc976dba06d3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:50.432127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 107/209] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:50.440684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1028-1928-4e90-992f-0ceb7bd5f436 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:50.440684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1028-1928-4e90-992f-0ceb7bd5f436 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:50.440684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1028-1928-4e90-992f-0ceb7bd5f436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:50.457459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e45b1028-1928-4e90-992f-0ceb7bd5f436 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:50.458912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e45b1028-1928-4e90-992f-0ceb7bd5f436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:50.502335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f374e-aa8f-443f-a013-1c97c54a2049 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:50.502640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f374e-aa8f-443f-a013-1c97c54a2049 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:50.502886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f374e-aa8f-443f-a013-1c97c54a2049 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:50.504406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 108/210] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:50.509229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4bbc-68ce-4716-9201-58ca4b8f0b48 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:50.509537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4bbc-68ce-4716-9201-58ca4b8f0b48 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:50.509770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4bbc-68ce-4716-9201-58ca4b8f0b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:50.528394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7c3b4bbc-68ce-4716-9201-58ca4b8f0b48 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:50.529588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7c3b4bbc-68ce-4716-9201-58ca4b8f0b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:50.532018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 103/211] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:50.555099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:50.609368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 109/212] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:50.816662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b1028-1928-4e90-992f-0ceb7bd5f436 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:50.892417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4bbc-68ce-4716-9201-58ca4b8f0b48 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:50.921703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b1028-1928-4e90-992f-0ceb7bd5f436 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:50.950100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:50.958380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d77846-08a2-4e8e-a411-96c489df27d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:50.959673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77846-08a2-4e8e-a411-96c489df27d6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:50.960164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77846-08a2-4e8e-a411-96c489df27d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:50.960616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77846-08a2-4e8e-a411-96c489df27d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:50.971205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d77846-08a2-4e8e-a411-96c489df27d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-a4d77846-08a2-4e8e-a411-96c489df27d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:50.976022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:50.976889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:50.977664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 104/213] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:48:50.983920 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61336b-8b7b-4609-82da-aab6a7e285cf None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:50.984140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61336b-8b7b-4609-82da-aab6a7e285cf None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:50.984410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61336b-8b7b-4609-82da-aab6a7e285cf None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:50.987530 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 105/214] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:50.987530 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61336b-8b7b-4609-82da-aab6a7e285cf None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.003798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4bbc-68ce-4716-9201-58ca4b8f0b48 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.037541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14caa23-8665-4538-be61-c2c610d5195d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.038177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14caa23-8665-4538-be61-c2c610d5195d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.038459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14caa23-8665-4538-be61-c2c610d5195d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.038725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14caa23-8665-4538-be61-c2c610d5195d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.039862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14caa23-8665-4538-be61-c2c610d5195d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:51.041968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14caa23-8665-4538-be61-c2c610d5195d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:51.043579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 110/215] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:51.044730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56054-eb82-4ac7-8c64-9c8ca7bfa471 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.045332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56054-eb82-4ac7-8c64-9c8ca7bfa471 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.045617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56054-eb82-4ac7-8c64-9c8ca7bfa471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.063481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-50b56054-eb82-4ac7-8c64-9c8ca7bfa471 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:51.064708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-50b56054-eb82-4ac7-8c64-9c8ca7bfa471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:51.084314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:51.111237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61336b-8b7b-4609-82da-aab6a7e285cf None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.154042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.154655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.154937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.155365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 111/216] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:51.155470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.156060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56054-eb82-4ac7-8c64-9c8ca7bfa471 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.160227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:51.160537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:51.161938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:51.174422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 106/217] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:51.180981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.237624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:51.281714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56054-eb82-4ac7-8c64-9c8ca7bfa471 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.297979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e69db-6fac-4bbf-ad0f-800e04470c3d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.325495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.326554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.327386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.327874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.328877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:51.330519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:51.339955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.340590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.340871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.341146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] PATH_INFO: `/v3/users/981cfed6219e4f59b4a0de8b45145115` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.356654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=981cfed6219e4f59b4a0de8b45145115)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:51.358124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:51.359402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 112/218] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:48:51 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:51.364653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 107/219] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:51 2022] GET /identity/v3/users/981cfed6219e4f59b4a0de8b45145115 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:51.371909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.388656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:51.425085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.496756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2894d2-22b6-46fe-81c3-04467111cd79 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.543552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d26ae-1a33-42fa-afe9-797ab6dfa4b4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.549369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76b57-b69d-47df-a88e-e942abfcfa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.550324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76b57-b69d-47df-a88e-e942abfcfa34 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.550802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76b57-b69d-47df-a88e-e942abfcfa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.551290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76b57-b69d-47df-a88e-e942abfcfa34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.554973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:51.595539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.596573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.597075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.597640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.614120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:51.619829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:51.620772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:51.621567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 108/220] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:51.626829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.643039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e76b57-b69d-47df-a88e-e942abfcfa34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:51.646350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e76b57-b69d-47df-a88e-e942abfcfa34 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:51.657786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:51.769192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e85b-f779-4dbf-b2aa-40416ec6d782 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.777446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 113/221] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:51.778866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70951c3-374f-41c5-a916-297960c5d012 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.779154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70951c3-374f-41c5-a916-297960c5d012 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.779428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70951c3-374f-41c5-a916-297960c5d012 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.780894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 114/222] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:51.784377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70951c3-374f-41c5-a916-297960c5d012 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.820687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d26359-42f8-4dfe-a5d5-9b9d3c61d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.821305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26359-42f8-4dfe-a5d5-9b9d3c61d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.821582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26359-42f8-4dfe-a5d5-9b9d3c61d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.821848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26359-42f8-4dfe-a5d5-9b9d3c61d7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.823029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26359-42f8-4dfe-a5d5-9b9d3c61d7e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:51.823325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26359-42f8-4dfe-a5d5-9b9d3c61d7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:51.824744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26359-42f8-4dfe-a5d5-9b9d3c61d7e9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:51.833378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 109/223] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:51.834516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ad4f1-b1e9-4be3-8644-d2aebf0d3fc5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.834843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ad4f1-b1e9-4be3-8644-d2aebf0d3fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.835063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ad4f1-b1e9-4be3-8644-d2aebf0d3fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.862941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-670ad4f1-b1e9-4be3-8644-d2aebf0d3fc5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:51.865018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-670ad4f1-b1e9-4be3-8644-d2aebf0d3fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:51.900195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:51.911666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70951c3-374f-41c5-a916-297960c5d012 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:51.925131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:51.956840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:51.958047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:51.958507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:51.959060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:51.960611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:51.961448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 110/224] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:51.962570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:51.965348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 115/225] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:51 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:51.968287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ad4f1-b1e9-4be3-8644-d2aebf0d3fc5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:51.971010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:52.150181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ad4f1-b1e9-4be3-8644-d2aebf0d3fc5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:52.185243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b021f-7e49-415d-aa5c-c70460b2e102 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:52.191536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:52.191536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:52.191536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:52.191536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] PATH_INFO: `/v3/users/49d38627f8d04829871a044f09ddf75b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:52.191870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] RBAC: Authorizing `identity:get_user(user_id=49d38627f8d04829871a044f09ddf75b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:52.192371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:52.195241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 111/226] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:51 2022] GET /identity/v3/users/49d38627f8d04829871a044f09ddf75b => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:52.203511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:52.238233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:52.238884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:52.239166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:52.239439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:52.240197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=981cfed6219e4f59b4a0de8b45145115) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:52.240535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:52.241986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:52.251434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 116/227] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:51 2022] GET /identity/v3/role_assignments?user.id=981cfed6219e4f59b4a0de8b45145115&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:52.251434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:52.266618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:52.306200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:52.392027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2340fb-90d0-4022-8950-1a5af0c52ecd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:52.409798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a8c14-e875-4cee-bef5-a8ebaed2e5a4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:52.451854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:52.452512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:52.452805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:52.453067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:52.454664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:52.456183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:52.460684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:52.461343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:52.461625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:52.461898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:52.467482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 112/228] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:52 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:52.468990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:52.495829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:52.501525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:52.502847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:52.503654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 117/229] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:48:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:52.511959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:52.607437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:52.614888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b843990-2682-41bb-ab85-cfdd6b130016 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:52.645690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aab116-8e7c-46ec-bf8b-b6f0f2e4030e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:52.647721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:52.654461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:52.655133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:52.655418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:52.655725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/49d38627f8d04829871a044f09ddf75b/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:52.664970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] RBAC: Authorizing `identity:create_grant(project_id=eb87cf3a34c846e8baad667e5145e480, user_id=49d38627f8d04829871a044f09ddf75b, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:52.667034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:52.692237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:52.693183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:52.693657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:52.694286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:52.695218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:52.695758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:52.698265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:52.707137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 118/230] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:48:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:52.711797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 113/231] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:52 2022] PUT /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/49d38627f8d04829871a044f09ddf75b/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:48:52.718718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:52.850637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc90486-3e1f-4e38-a257-c22483455a1a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:52.905578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:52.906632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:52.907219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:52.907727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] PATH_INFO: `/v3/users/49d38627f8d04829871a044f09ddf75b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:52.909332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] RBAC: Authorizing `identity:get_user(user_id=49d38627f8d04829871a044f09ddf75b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:52.911076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:52.913113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 119/232] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:52 2022] GET /identity/v3/users/49d38627f8d04829871a044f09ddf75b => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:52.919287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:53.041735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e44ce7-1c94-4c5c-a92c-2aebb30d22ad None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:53.086063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12a250-fe63-496c-86ab-90b3443020ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:53.086706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12a250-fe63-496c-86ab-90b3443020ca None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:53.086988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12a250-fe63-496c-86ab-90b3443020ca None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:53.087258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12a250-fe63-496c-86ab-90b3443020ca None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:53.089872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12a250-fe63-496c-86ab-90b3443020ca None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:53.091344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12a250-fe63-496c-86ab-90b3443020ca None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:53.093170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 114/233] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:52 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:53.099128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:53.147543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:53.238352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d05ba2-2ade-420a-b24e-9dd4a038aa9f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:53.282602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff61f0-6c78-4700-9439-403f8eca2308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:53.283280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff61f0-6c78-4700-9439-403f8eca2308 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:53.283585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff61f0-6c78-4700-9439-403f8eca2308 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:53.283857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff61f0-6c78-4700-9439-403f8eca2308 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/49d38627f8d04829871a044f09ddf75b/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:53.286092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff61f0-6c78-4700-9439-403f8eca2308 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb87cf3a34c846e8baad667e5145e480, user_id=49d38627f8d04829871a044f09ddf75b, role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:53.287587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff61f0-6c78-4700-9439-403f8eca2308 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:53.311173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 120/234] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:53 2022] PUT /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/49d38627f8d04829871a044f09ddf75b/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:48:53.988184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83bb4-b83b-4789-98ff-16e21a08358f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:53.988705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83bb4-b83b-4789-98ff-16e21a08358f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:53.988705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83bb4-b83b-4789-98ff-16e21a08358f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:53.990237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 115/235] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:53.996371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ecbe2-ae86-4f93-be21-787a67b1992b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:53.996857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ecbe2-ae86-4f93-be21-787a67b1992b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:53.996956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ecbe2-ae86-4f93-be21-787a67b1992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:54.020498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4c3ecbe2-ae86-4f93-be21-787a67b1992b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:54.021715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4c3ecbe2-ae86-4f93-be21-787a67b1992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:54.109935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 121/236] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:54.292517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f440-1d3e-456f-bc71-888dc303ce00 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:54.292892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f440-1d3e-456f-bc71-888dc303ce00 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:54.293067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f440-1d3e-456f-bc71-888dc303ce00 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:54.294563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 116/237] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:54.307891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b034-401c-4457-b2e0-ce2d1c7d2719 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:54.308182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b034-401c-4457-b2e0-ce2d1c7d2719 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:54.308458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b034-401c-4457-b2e0-ce2d1c7d2719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:54.334568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0665b034-401c-4457-b2e0-ce2d1c7d2719 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:54.336475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0665b034-401c-4457-b2e0-ce2d1c7d2719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:54.359682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:54.425437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 122/238] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:54.575592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f440-1d3e-456f-bc71-888dc303ce00 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:54.606198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:54.714910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f440-1d3e-456f-bc71-888dc303ce00 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:54.762551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:54.763527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:54.764036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:54.764515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:54.768183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:54.769629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:54.771371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 117/239] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:54.776896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b034-401c-4457-b2e0-ce2d1c7d2719 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:54.783464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:54.898929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b034-401c-4457-b2e0-ce2d1c7d2719 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:54.944166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:54.944803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:54.945083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:54.945356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:54.946032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:54.948116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:54.967896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:54.979264 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6283f-c410-47ce-a779-5f877ce4f5aa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:55.001137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 123/240] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:48:54 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:55.024305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23f63-aef8-441c-bbb0-0f6e9ae3aa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:55.025620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23f63-aef8-441c-bbb0-0f6e9ae3aa96 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.026041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23f63-aef8-441c-bbb0-0f6e9ae3aa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.026452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23f63-aef8-441c-bbb0-0f6e9ae3aa96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.038397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff23f63-aef8-441c-bbb0-0f6e9ae3aa96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5ff23f63-aef8-441c-bbb0-0f6e9ae3aa96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:55.043810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:55.045175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:48:55.046162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:55.048123 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 118/241] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:48:55.057107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:55.071522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:55.072022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a41772-3d91-46b5-9072-18469731cc31 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.072538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a41772-3d91-46b5-9072-18469731cc31 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.072952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a41772-3d91-46b5-9072-18469731cc31 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.075898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 119/242] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:48:55.081482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bdd47-d96a-4ba0-84bd-6efa4ad7c5b0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.082001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bdd47-d96a-4ba0-84bd-6efa4ad7c5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.082410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bdd47-d96a-4ba0-84bd-6efa4ad7c5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.102581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-750bdd47-d96a-4ba0-84bd-6efa4ad7c5b0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:55.104185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-750bdd47-d96a-4ba0-84bd-6efa4ad7c5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:55.232664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 120/243] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:55.263151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc46acf-0393-4a5e-bb46-1899eecc551b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:55.304986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f28cb-5936-4e6e-ad1d-9b87bd5bbe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:55.305839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f28cb-5936-4e6e-ad1d-9b87bd5bbe2d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.306257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f28cb-5936-4e6e-ad1d-9b87bd5bbe2d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.307184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f28cb-5936-4e6e-ad1d-9b87bd5bbe2d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.308043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f28cb-5936-4e6e-ad1d-9b87bd5bbe2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:55.308482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f28cb-5936-4e6e-ad1d-9b87bd5bbe2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:55.310009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f28cb-5936-4e6e-ad1d-9b87bd5bbe2d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:55.319029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 124/244] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:55.326265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bdd47-d96a-4ba0-84bd-6efa4ad7c5b0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:55.350062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63bcd9-ae59-4ea4-99ab-5895620f2d50 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.350563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63bcd9-ae59-4ea4-99ab-5895620f2d50 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.350987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63bcd9-ae59-4ea4-99ab-5895620f2d50 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.353227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 125/245] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:55.359182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3a17f-4a64-4d5b-9984-e52932252c0a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.359711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3a17f-4a64-4d5b-9984-e52932252c0a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.360214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3a17f-4a64-4d5b-9984-e52932252c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.380849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-39f3a17f-4a64-4d5b-9984-e52932252c0a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:55.382049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-39f3a17f-4a64-4d5b-9984-e52932252c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:55.429233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:55.458052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bdd47-d96a-4ba0-84bd-6efa4ad7c5b0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:55.458913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:55.466579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 126/246] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:55.498307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:55.498880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.499194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.499467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] PATH_INFO: `/v3/users/6aca958b0b8d492396a6dfc360860708` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.513596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] RBAC: Authorizing `identity:get_user(user_id=6aca958b0b8d492396a6dfc360860708)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:55.515041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:55.516702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 121/247] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:55 2022] GET /identity/v3/users/6aca958b0b8d492396a6dfc360860708 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:55.522340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3a17f-4a64-4d5b-9984-e52932252c0a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:55.580156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:55.644836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3a17f-4a64-4d5b-9984-e52932252c0a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:55.690343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:55.691608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.692085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.692626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.719111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c26b95-fafd-4420-b8bb-2fca0330b5b2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:55.770840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:55.771779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.772198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.772608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.785662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:55.786713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:55.793796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:55.795107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:55.796019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:55.797008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 122/248] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:48:55.797574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:55.803657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:55.808059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:55.811882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:55.920755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bba4cf-eae4-434b-a793-f5deed40824b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:55.930980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 127/249] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:55.932833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:55.966205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:55.966846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:55.967121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:55.967392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:55.979041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:55.984019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:55.984911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:55.986078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 123/250] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:48:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:55.987021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.052159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:56.056153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a8843-301f-445b-8a22-75ffc9d8ccba None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:56.075265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:56.136512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:56.137467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.137927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.138377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.139219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:56.140692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:56.142293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:56.156790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 128/251] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:56.157245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56442b-a5ba-4762-aab8-e7fdd45f5627 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:56.157894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.199637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add44063-ad66-408a-a21b-58ae15d2a719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:56.200553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add44063-ad66-408a-a21b-58ae15d2a719 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.200977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add44063-ad66-408a-a21b-58ae15d2a719 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.201390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add44063-ad66-408a-a21b-58ae15d2a719 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.202675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add44063-ad66-408a-a21b-58ae15d2a719 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:56.204742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add44063-ad66-408a-a21b-58ae15d2a719 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:56.207580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 124/252] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:55 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:56.209036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543de77-9b42-4b2f-851c-7e6be2706cdb None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.209493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543de77-9b42-4b2f-851c-7e6be2706cdb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.209938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543de77-9b42-4b2f-851c-7e6be2706cdb None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.211323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 125/253] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:56.216278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543de77-9b42-4b2f-851c-7e6be2706cdb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.268644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de63de-44d2-4b38-8e3c-1cd02f95d29b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:56.305149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:56.305791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.306146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.306495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.307309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:56.307729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:56.309243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:56.323619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 129/254] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:56.324700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.325915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543de77-9b42-4b2f-851c-7e6be2706cdb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:56.354249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:56.354702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:56.367227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ad78-b2ce-4947-9a88-cfff85c9d7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:56.368055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ad78-b2ce-4947-9a88-cfff85c9d7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.368343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ad78-b2ce-4947-9a88-cfff85c9d7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.368621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ad78-b2ce-4947-9a88-cfff85c9d7c0 None admin] PATH_INFO: `/v3/users/981cfed6219e4f59b4a0de8b45145115` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.370694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3ad78-b2ce-4947-9a88-cfff85c9d7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=981cfed6219e4f59b4a0de8b45145115)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:56.372254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3ad78-b2ce-4947-9a88-cfff85c9d7c0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:56.374032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 126/255] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:56 2022] GET /identity/v3/users/981cfed6219e4f59b4a0de8b45145115 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:56.375737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea766e-1e97-43db-996e-66c940f1047a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.376030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea766e-1e97-43db-996e-66c940f1047a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.376308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea766e-1e97-43db-996e-66c940f1047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.398144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8fea766e-1e97-43db-996e-66c940f1047a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:56.399726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8fea766e-1e97-43db-996e-66c940f1047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:56.444812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5eea-1940-4b4a-a452-c9b4f08ecc50 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:56.477874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 127/256] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:56.479274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea766e-1e97-43db-996e-66c940f1047a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.489079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:56.490045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.490551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.491022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.491952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] RBAC: Adding query filter params (role.id=78d8f7f202354798acd7876e3bf7deff, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=6aca958b0b8d492396a6dfc360860708) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:56.492437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:56.494128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:56.509056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 130/257] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:56 2022] GET /identity/v3/role_assignments?user.id=6aca958b0b8d492396a6dfc360860708&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=78d8f7f202354798acd7876e3bf7deff => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:56.510613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.659602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea766e-1e97-43db-996e-66c940f1047a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:56.715864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:56.748785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:56.759935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:56.760602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.760888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.761174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] PATH_INFO: `/v3/users/49d38627f8d04829871a044f09ddf75b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.762344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] RBAC: Authorizing `identity:get_user(user_id=49d38627f8d04829871a044f09ddf75b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:56.763298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb4ea4-7e6c-4cff-8d42-e283aa40ea9a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:56.763812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:56.765394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 128/258] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:56 2022] GET /identity/v3/users/49d38627f8d04829871a044f09ddf75b => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:56.771559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.821970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:56.823022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:56.823477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:56.823947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:56.825531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:56.827872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:56.834441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 131/259] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:56 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:56.835720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:56.980861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe910b9e-6129-4e44-8e73-fb41f39381cd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:57.022271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffd981-d223-4d63-a535-bad3a2c586d5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:57.052837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:57.053460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:57.053743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:57.054014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:57.084775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:57.085455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:57.087383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:57.087697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/981cfed6219e4f59b4a0de8b45145115/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:57.091804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] RBAC: Authorizing `identity:create_grant(project_id=eb87cf3a34c846e8baad667e5145e480, user_id=981cfed6219e4f59b4a0de8b45145115, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:57.093281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:57.120616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 132/260] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:48:56 2022] PUT /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/981cfed6219e4f59b4a0de8b45145115/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:48:57.121510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:57.157117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:57.173148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:57.191115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:57.192789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:57.297607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d10a-5cd8-4bf6-bfea-b91a7a670bac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:57.353161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 129/261] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 22:48:57.358966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:57.373238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:57.374262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:57.374766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:57.375285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:57.395683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:57.406905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:57.407862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:48:57.408638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:48:57.411340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 133/262] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:48:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:48:57.417548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:57.519175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:57.531196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bced20-16d8-433c-b3ad-7dcd470a0cd9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:57.542289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:57.571020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29e62-ba8e-4104-96fe-dc874724ac10 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:57.588784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:57.589436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:57.589718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:57.589988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:57.591163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:57.592643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:57.595268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 130/263] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:57 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:57.600078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:57.610618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:57.611589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:57.612064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:57.612556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:57.613522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:57.614072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:57.615681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:57.625664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 134/264] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:48:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:57.632508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:57.821520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd631ea9-0319-465b-97cc-44a191233907 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:57.879190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd4866-079b-4fe0-974c-5774e8b6c30b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:57.906247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:57.906868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:57.907149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:57.907422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:57.908240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=49d38627f8d04829871a044f09ddf75b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:57.908484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:57.909911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:57.925662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 131/265] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:57 2022] GET /identity/v3/role_assignments?user.id=49d38627f8d04829871a044f09ddf75b&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:57.968476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5662650-2ef2-4e5e-98c6-43d84b73f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:57.969167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662650-2ef2-4e5e-98c6-43d84b73f942 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:57.969454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662650-2ef2-4e5e-98c6-43d84b73f942 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:57.969734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662650-2ef2-4e5e-98c6-43d84b73f942 None admin] PATH_INFO: `/v3/users/49d38627f8d04829871a044f09ddf75b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:57.971124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5662650-2ef2-4e5e-98c6-43d84b73f942 None admin] RBAC: Authorizing `identity:get_user(user_id=49d38627f8d04829871a044f09ddf75b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:57.972700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5662650-2ef2-4e5e-98c6-43d84b73f942 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:57.974442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 135/266] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:57 2022] GET /identity/v3/users/49d38627f8d04829871a044f09ddf75b => generated 283 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:57.980985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:58.033475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:58.079847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c89cfd-2a5d-420b-8a38-ad8ea84e43d9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.080130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c89cfd-2a5d-420b-8a38-ad8ea84e43d9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.080312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c89cfd-2a5d-420b-8a38-ad8ea84e43d9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.081649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 136/267] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:58.088860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db70fd-f3db-43da-bed7-145bcfd43d19 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.089479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db70fd-f3db-43da-bed7-145bcfd43d19 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.089939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db70fd-f3db-43da-bed7-145bcfd43d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.105455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:58.113937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-79db70fd-f3db-43da-bed7-145bcfd43d19 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:58.115378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-79db70fd-f3db-43da-bed7-145bcfd43d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:58.197562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 137/268] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:58.209794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869a1f9-2d94-43a2-8306-a5f60d899ebb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:58.269098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:58.269728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.270014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.270291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.363587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:58.364987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:58.400886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:58.496078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 132/269] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:48:58.501634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db70fd-f3db-43da-bed7-145bcfd43d19 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:58.589886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:58.650744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db70fd-f3db-43da-bed7-145bcfd43d19 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:58.698439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:58.699458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.700094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.700556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:58.704193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:58.706704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 138/270] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:48:58 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:58.713006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:58.721353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:58.774820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db5302-645a-4583-b77f-7c59bc71b1c7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:58.805986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:58.820435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8f182-0792-4b29-9903-4e2a990fc567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:58.821489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8f182-0792-4b29-9903-4e2a990fc567 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.821965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8f182-0792-4b29-9903-4e2a990fc567 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.822545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8f182-0792-4b29-9903-4e2a990fc567 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.840089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8f182-0792-4b29-9903-4e2a990fc567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-31c8f182-0792-4b29-9903-4e2a990fc567 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:48:58.845604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:48:58.846502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:48:58.847286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:48:58.849971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 133/271] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:48:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:48:58.851348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c37e-ae9f-41be-8bcf-22c79b663a18 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.856144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c37e-ae9f-41be-8bcf-22c79b663a18 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.856668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c37e-ae9f-41be-8bcf-22c79b663a18 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.859218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 134/272] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:58.860894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c37e-ae9f-41be-8bcf-22c79b663a18 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:58.868348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1085e-a341-4636-b71f-b50662fe0cbe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:58.903798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f583d10-6f8c-4de6-8b84-1d62488ab658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:58.904335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f583d10-6f8c-4de6-8b84-1d62488ab658 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.904615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f583d10-6f8c-4de6-8b84-1d62488ab658 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.904889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f583d10-6f8c-4de6-8b84-1d62488ab658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.905612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f583d10-6f8c-4de6-8b84-1d62488ab658 None admin] RBAC: Adding query filter params (role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=49d38627f8d04829871a044f09ddf75b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:58.905935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f583d10-6f8c-4de6-8b84-1d62488ab658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:58.907370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f583d10-6f8c-4de6-8b84-1d62488ab658 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:58.913630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 139/273] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:48:58 2022] GET /identity/v3/role_assignments?user.id=49d38627f8d04829871a044f09ddf75b&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:58.914938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267707-74e0-40c7-92a8-81891be29f64 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:58.915223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267707-74e0-40c7-92a8-81891be29f64 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:58.915511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267707-74e0-40c7-92a8-81891be29f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:58.935945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-35267707-74e0-40c7-92a8-81891be29f64 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:58.937153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35267707-74e0-40c7-92a8-81891be29f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:58.981569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c37e-ae9f-41be-8bcf-22c79b663a18 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:59.030705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:59.032175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.032946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.034154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.035045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:59.035784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:59.038281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:59.057023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 135/274] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:48:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:48:59.064268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:59.073926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:59.085159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 140/275] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:48:59.127022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:59.179035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da13b0a-db04-45cc-bb03-2e6b7661926a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:59.213024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:59.213804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.213992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.214267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] PATH_INFO: `/v3/users/7b03d1d47d934d9eb0b5e10400d3543e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.228065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] RBAC: Authorizing `identity:get_user(user_id=7b03d1d47d934d9eb0b5e10400d3543e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:59.230050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:59.232041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 136/276] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:48:59 2022] GET /identity/v3/users/7b03d1d47d934d9eb0b5e10400d3543e => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:59.240626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35267707-74e0-40c7-92a8-81891be29f64 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:59.362582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35267707-74e0-40c7-92a8-81891be29f64 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:59.402388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:59.403002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.403277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.403572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.404428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, user.id=7b03d1d47d934d9eb0b5e10400d3543e, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:48:59.404730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:59.406232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:59.411029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:59.413171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 141/277] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 22:48:59 2022] GET /identity/v3/role_assignments?user.id=7b03d1d47d934d9eb0b5e10400d3543e&scope.system=all&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:59.529099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941bf42-d4dd-420c-894e-efff6a9f5acb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:59.538092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:59.574463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17942222-a191-43df-87ff-f4f86babc8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:59.575106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17942222-a191-43df-87ff-f4f86babc8df None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.575383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17942222-a191-43df-87ff-f4f86babc8df None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.575681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17942222-a191-43df-87ff-f4f86babc8df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.576828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17942222-a191-43df-87ff-f4f86babc8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:59.578326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17942222-a191-43df-87ff-f4f86babc8df None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:59.580373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 137/278] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:48:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:48:59.586175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:48:59.629299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac65b8-4f34-4fe2-b978-f5028a7cc9d7 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.629689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac65b8-4f34-4fe2-b978-f5028a7cc9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.629963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac65b8-4f34-4fe2-b978-f5028a7cc9d7 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.631298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 138/279] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:59.636515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669b0f-fce8-481e-af31-fbebc1f71788 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.636795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669b0f-fce8-481e-af31-fbebc1f71788 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.637064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669b0f-fce8-481e-af31-fbebc1f71788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.659452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-64669b0f-fce8-481e-af31-fbebc1f71788 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:59.660650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-64669b0f-fce8-481e-af31-fbebc1f71788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:48:59.663625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:48:59.716036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea4002-2064-4613-bf5a-c09a4829dab5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:48:59.760649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 139/280] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:48:59.770899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01296531-bd54-469e-b320-da2788908cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:48:59.771865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01296531-bd54-469e-b320-da2788908cd8 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.772344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01296531-bd54-469e-b320-da2788908cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.772853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01296531-bd54-469e-b320-da2788908cd8 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.773736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01296531-bd54-469e-b320-da2788908cd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:48:59.775912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01296531-bd54-469e-b320-da2788908cd8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:48:59.800473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 142/281] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:48:59 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:48:59.958142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1ba09-3bae-4bdb-8b4d-61cb6b6ad92f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.958880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1ba09-3bae-4bdb-8b4d-61cb6b6ad92f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.959361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1ba09-3bae-4bdb-8b4d-61cb6b6ad92f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.961366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 140/282] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:48:59.968059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27c0a-6c0f-4b4c-b713-0dbb48f63e27 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:48:59.968653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27c0a-6c0f-4b4c-b713-0dbb48f63e27 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:48:59.969118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27c0a-6c0f-4b4c-b713-0dbb48f63e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:48:59.989358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f6a27c0a-6c0f-4b4c-b713-0dbb48f63e27 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:48:59.990930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f6a27c0a-6c0f-4b4c-b713-0dbb48f63e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:00.042113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1ba09-3bae-4bdb-8b4d-61cb6b6ad92f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:00.069239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 143/283] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:00.164691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1ba09-3bae-4bdb-8b4d-61cb6b6ad92f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:00.167709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:00.207153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:00.207828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:00.208106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:00.208375 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:00.219517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:00.224315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:00.225195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:00.226033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 141/284] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:00.231999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a27c0a-6c0f-4b4c-b713-0dbb48f63e27 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:00.297380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:00.351861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a27c0a-6c0f-4b4c-b713-0dbb48f63e27 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:00.355969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:00.395552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:00.396219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:00.396495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:00.396768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:00.397481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:00.397774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:00.399407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:00.412817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 144/285] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:00.414399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:00.502627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb90e-6528-41b4-82dc-e9eec94fc269 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:00.541901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:00.543068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:00.543583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:00.544092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:00.557223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:00.562540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:00.563452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:00.564295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:00.575239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 142/286] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:00.576458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:00.617449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd6dcb-9bb7-4ccb-8cd8-e5efa8b90b84 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:00.632204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:00.645551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:00.709560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:00.710201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:00.710478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:00.717801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] PATH_INFO: `/v3/users/6aca958b0b8d492396a6dfc360860708` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:00.718395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360d250-6937-48e1-bae6-0fbdd1bd902a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:00.719206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=6aca958b0b8d492396a6dfc360860708)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:00.720708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:00.722320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 145/287] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:00 2022] GET /identity/v3/users/6aca958b0b8d492396a6dfc360860708 => generated 297 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:00.732772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:00.760083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:00.760640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:00.760916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:00.761183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:00.761843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:00.762135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:00.763524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:00.783274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 143/288] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:00.790292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:00.934992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96c54e-3a9e-4415-972e-cdf991832fb4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:00.943837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:00.963361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef5d9-6d53-4d88-b48f-9efe96a80c49 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:01.001737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:01.003704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.004355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.005795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.007287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:01.010902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:01.015145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 146/289] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:00 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:01.018559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:01.031800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:01.032785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.033283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.033768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] PATH_INFO: `/v3/users/981cfed6219e4f59b4a0de8b45145115` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.035124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] RBAC: Authorizing `identity:get_user(user_id=981cfed6219e4f59b4a0de8b45145115)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:01.038136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:01.047964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 144/290] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:00 2022] GET /identity/v3/users/981cfed6219e4f59b4a0de8b45145115 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:01.054522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:01.123959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:01.184783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e261-ee7d-426c-a9d7-6d8fe7e4403c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:01.255111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9f925-804a-42ab-9185-c3b11d3bdbdd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:01.294740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:01.296386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.297847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:01.298354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.299185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/6aca958b0b8d492396a6dfc360860708/roles/78d8f7f202354798acd7876e3bf7deff` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.300817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.301303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.302281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.303685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb87cf3a34c846e8baad667e5145e480, user_id=6aca958b0b8d492396a6dfc360860708, role_id=78d8f7f202354798acd7876e3bf7deff)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:01.306008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:01.336120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 147/291] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:49:01 2022] PUT /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480/users/6aca958b0b8d492396a6dfc360860708/roles/78d8f7f202354798acd7876e3bf7deff => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:01.349280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:01.415753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:01.417285 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:01.561708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 145/292] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 22:49:01.567584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:01.682323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca7619-97a7-4b90-b77c-5b1b2cc72182 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:01.689671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:01.725338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501de60-e430-4595-bb8c-3ba9621859ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:01.726012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501de60-e430-4595-bb8c-3ba9621859ee None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.726294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501de60-e430-4595-bb8c-3ba9621859ee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.726568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501de60-e430-4595-bb8c-3ba9621859ee None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.730914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501de60-e430-4595-bb8c-3ba9621859ee None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:01.730914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501de60-e430-4595-bb8c-3ba9621859ee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:01.732126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 148/293] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:01 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:01.740967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:01.819576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:01.831014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d00a0-26e2-4130-816c-59377a9b3878 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.831312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d00a0-26e2-4130-816c-59377a9b3878 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.831623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d00a0-26e2-4130-816c-59377a9b3878 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.833070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 149/294] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:01.838499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20b463-d685-4018-a091-b0df3e710d96 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.838781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20b463-d685-4018-a091-b0df3e710d96 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.839054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20b463-d685-4018-a091-b0df3e710d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.873849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da2fc1-b168-4235-8016-9bf72ad6b8c6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:01.876337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0a20b463-d685-4018-a091-b0df3e710d96 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:01.877529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0a20b463-d685-4018-a091-b0df3e710d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:01.941174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:01.941786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:01.942064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:01.942334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:01.943047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=981cfed6219e4f59b4a0de8b45145115) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:01.943352 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:01.944818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:01.958515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 146/295] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:01 2022] GET /identity/v3/role_assignments?user.id=981cfed6219e4f59b4a0de8b45145115&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:01.988279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 150/296] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:02.350952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:02.403235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeef793-22a6-43b1-9d1a-30b8f1e92d43 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:02.403814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeef793-22a6-43b1-9d1a-30b8f1e92d43 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:02.403814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeef793-22a6-43b1-9d1a-30b8f1e92d43 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:02.405405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 151/297] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:02.410500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b4645-1673-4248-8d7d-7df2ddec41fa None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:02.410795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b4645-1673-4248-8d7d-7df2ddec41fa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:02.411071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b4645-1673-4248-8d7d-7df2ddec41fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:02.435698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a18b4645-1673-4248-8d7d-7df2ddec41fa None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:02.436762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a18b4645-1673-4248-8d7d-7df2ddec41fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:02.471224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f987-9ff9-4f01-8650-9cd9354538ab None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:02.499698 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:02.530453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:02.531129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:02.531407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:02.531719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:02.540200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 152/298] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:02.551516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:02.553964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:02.554829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:02.555977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 147/299] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:02.561455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b4645-1673-4248-8d7d-7df2ddec41fa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:02.608575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:02.679146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b4645-1673-4248-8d7d-7df2ddec41fa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:02.716716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:02.717662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:02.718142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:02.718634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:02.719528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:02.720032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:02.721602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:02.731059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 153/300] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:02.736911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:02.836978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:02.851440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1837da-7155-47cc-89f2-9625ae13e223 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:02.892435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e41942-1443-42dd-a240-e7db76d61907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:02.892992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e41942-1443-42dd-a240-e7db76d61907 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:02.893277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e41942-1443-42dd-a240-e7db76d61907 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:02.893553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e41942-1443-42dd-a240-e7db76d61907 None admin] PATH_INFO: `/v3/users/7b03d1d47d934d9eb0b5e10400d3543e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:02.894688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e41942-1443-42dd-a240-e7db76d61907 None admin] RBAC: Authorizing `identity:get_user(user_id=7b03d1d47d934d9eb0b5e10400d3543e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:02.896766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e41942-1443-42dd-a240-e7db76d61907 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:02.898516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 148/301] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:02 2022] GET /identity/v3/users/7b03d1d47d934d9eb0b5e10400d3543e => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:02.899545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451bdc-d7fc-4023-a1e9-3e6af98d5663 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:02.899834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451bdc-d7fc-4023-a1e9-3e6af98d5663 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:02.900697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451bdc-d7fc-4023-a1e9-3e6af98d5663 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:02.902161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 149/302] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:02.905702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba451bdc-d7fc-4023-a1e9-3e6af98d5663 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:02.941786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:02.948806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e2966-b86c-4c7a-82c7-d0959880dde6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:02.956919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:02.991536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bd3ba-adbc-46b9-932b-610e50577ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:02.992326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd3ba-adbc-46b9-932b-610e50577ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:02.992605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd3ba-adbc-46b9-932b-610e50577ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:02.993231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd3ba-adbc-46b9-932b-610e50577ed9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.005226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bd3ba-adbc-46b9-932b-610e50577ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d75bd3ba-adbc-46b9-932b-610e50577ed9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:03.012547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:03.013393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:03.014185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 154/303] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:03.016119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba451bdc-d7fc-4023-a1e9-3e6af98d5663 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:03.020082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6d90-c3ae-463c-8bc9-0bb69de5fa90 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.020360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6d90-c3ae-463c-8bc9-0bb69de5fa90 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.020625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6d90-c3ae-463c-8bc9-0bb69de5fa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.037871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-91da6d90-c3ae-463c-8bc9-0bb69de5fa90 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:03.039052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-91da6d90-c3ae-463c-8bc9-0bb69de5fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:03.053678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:03.054323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.054598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.054866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] PATH_INFO: `/v3/system/users/7b03d1d47d934d9eb0b5e10400d3543e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.056391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7b03d1d47d934d9eb0b5e10400d3543e, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:03.057743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:03.066402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 150/304] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Sep 9 22:49:02 2022] PUT /identity/v3/system/users/7b03d1d47d934d9eb0b5e10400d3543e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:03.067012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:03.140856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 155/305] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:03.190434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86326a35-bb90-4af3-94ec-fab2f69b9e07 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:03.203367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:03.233439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:03.234428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.234904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.235398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.236601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:03.236900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:03.238345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:03.250249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 151/306] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:03.255794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da6d90-c3ae-463c-8bc9-0bb69de5fa90 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:03.347652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:03.363459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da6d90-c3ae-463c-8bc9-0bb69de5fa90 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:03.399433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:03.400623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.400903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.401173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] PATH_INFO: `/v3/users/1dca1c18cf484f879abde1bc588a2e06` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.415456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] RBAC: Authorizing `identity:get_user(user_id=1dca1c18cf484f879abde1bc588a2e06)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:03.417030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:03.419037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 156/307] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:03 2022] GET /identity/v3/users/1dca1c18cf484f879abde1bc588a2e06 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:03.425163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:03.428522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:03.534471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1963-c38a-4615-b5da-101a1c6a5e9a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:03.551830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c56eb2-f4a1-4327-a565-32fa66cf35f5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:03.569364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:03.570164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.570537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.570906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.573265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:03.574347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:03.576141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 157/308] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:03.581563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:03.593193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ca6fe-cf8f-47ff-853d-fca2feb85a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:03.594119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca6fe-cf8f-47ff-853d-fca2feb85a8f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.594408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca6fe-cf8f-47ff-853d-fca2feb85a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.594680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca6fe-cf8f-47ff-853d-fca2feb85a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.595701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ca6fe-cf8f-47ff-853d-fca2feb85a8f None admin] RBAC: Adding query filter params (role.id=78d8f7f202354798acd7876e3bf7deff, user.id=1dca1c18cf484f879abde1bc588a2e06, scope.system=all) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:03.596010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ca6fe-cf8f-47ff-853d-fca2feb85a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:03.597630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ca6fe-cf8f-47ff-853d-fca2feb85a8f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:03.598551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:03.606508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 152/309] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 22:49:03 2022] GET /identity/v3/role_assignments?user.id=1dca1c18cf484f879abde1bc588a2e06&scope.system=all&role.id=78d8f7f202354798acd7876e3bf7deff => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:03.707961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530bc2-8047-4ab2-b2f8-592facb3a874 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:03.757284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92639390-e81c-4769-93eb-f3a6659862ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:03.757915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92639390-e81c-4769-93eb-f3a6659862ee None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.758190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92639390-e81c-4769-93eb-f3a6659862ee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.758460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92639390-e81c-4769-93eb-f3a6659862ee None admin] PATH_INFO: `/v3/groups` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.759105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92639390-e81c-4769-93eb-f3a6659862ee None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:03.763212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92639390-e81c-4769-93eb-f3a6659862ee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:03.782391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 158/310] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Sep 9 22:49:03 2022] POST /identity/v3/groups => generated 229 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:49:03.936380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf715c-3585-4cea-942f-037f12f922cb None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.936681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf715c-3585-4cea-942f-037f12f922cb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.936906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf715c-3585-4cea-942f-037f12f922cb None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.939433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 153/311] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:03.945299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd5a7-0219-4b84-a7c2-4ef972fd0e9c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:03.945587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd5a7-0219-4b84-a7c2-4ef972fd0e9c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:03.945864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd5a7-0219-4b84-a7c2-4ef972fd0e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:03.965476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9a0dd5a7-0219-4b84-a7c2-4ef972fd0e9c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:03.966697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9a0dd5a7-0219-4b84-a7c2-4ef972fd0e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:04.047987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 159/312] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:04.338956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf715c-3585-4cea-942f-037f12f922cb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:04.379711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:04.448924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf715c-3585-4cea-942f-037f12f922cb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:04.482796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:04.483434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:04.483737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:04.484009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:04.494899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:04.500279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:04.501135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:04.501902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 154/313] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:04.507813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0dd5a7-0219-4b84-a7c2-4ef972fd0e9c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:04.525687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:04.611572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0dd5a7-0219-4b84-a7c2-4ef972fd0e9c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:04.650304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:04.650997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:04.651274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:04.651578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:04.652244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:04.652543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:04.653915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:04.662505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 160/314] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:04.668657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:04.778020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:04.783848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b523a25-f094-49ad-90da-5b0f5c908b4e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:04.819712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:04.820338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:04.820617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:04.820882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] PATH_INFO: `/v3/users/6aca958b0b8d492396a6dfc360860708` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:04.822057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] RBAC: Authorizing `identity:get_user(user_id=6aca958b0b8d492396a6dfc360860708)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:04.823501 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:04.825057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 155/315] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:04 2022] GET /identity/v3/users/6aca958b0b8d492396a6dfc360860708 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:04.830600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:04.899069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:04.939326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d376-17b9-4e57-917f-83443c718a02 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:04.973889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:04.974497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:04.974771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:04.975036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.049180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:05.051026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:05.117105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:05.156611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 161/316] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:05.162157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:05.267549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7484a-156b-44c6-9881-d4e8aad434ec None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:05.303550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86139b87-8ffe-4279-98a3-573ac543ed3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:05.304566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86139b87-8ffe-4279-98a3-573ac543ed3d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:05.305070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86139b87-8ffe-4279-98a3-573ac543ed3d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:05.305539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86139b87-8ffe-4279-98a3-573ac543ed3d None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.307100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86139b87-8ffe-4279-98a3-573ac543ed3d None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:05.308850 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86139b87-8ffe-4279-98a3-573ac543ed3d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:05.311232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 156/317] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:05 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:05.317608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:05.424427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a6a18-081c-4305-af6f-c93f9eacdd58 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:05.439303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227cf5d-470a-40f0-88f8-c95a6bdd8b74 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:05.440288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227cf5d-470a-40f0-88f8-c95a6bdd8b74 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:05.441040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227cf5d-470a-40f0-88f8-c95a6bdd8b74 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.442912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 157/318] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:05.448517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1a15-aeb3-4dee-b90a-ce32d82d694b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:05.449027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1a15-aeb3-4dee-b90a-ce32d82d694b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:05.449943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1a15-aeb3-4dee-b90a-ce32d82d694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.465307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:05.465912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:05.466184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:05.466448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.467169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] RBAC: Adding query filter params (role.id=78d8f7f202354798acd7876e3bf7deff, scope.project.id=eb87cf3a34c846e8baad667e5145e480, user.id=6aca958b0b8d492396a6dfc360860708) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:05.467471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:05.468935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:05.473908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bf9b1a15-aeb3-4dee-b90a-ce32d82d694b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:05.474966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 162/319] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:05 2022] GET /identity/v3/role_assignments?user.id=6aca958b0b8d492396a6dfc360860708&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=78d8f7f202354798acd7876e3bf7deff => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:05.475581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bf9b1a15-aeb3-4dee-b90a-ce32d82d694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:05.506799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:05.554223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 158/320] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:05.819130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:05.849579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:05.864202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce392d-4eec-4c9c-954e-1006babe2887 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:05.864495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce392d-4eec-4c9c-954e-1006babe2887 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:05.864765 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce392d-4eec-4c9c-954e-1006babe2887 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.866231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 159/321] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:05.871329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991d397-ea0f-49f0-aebb-5b07aec1fcb8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:05.871660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991d397-ea0f-49f0-aebb-5b07aec1fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:05.871946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991d397-ea0f-49f0-aebb-5b07aec1fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.893609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a991d397-ea0f-49f0-aebb-5b07aec1fcb8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:05.894826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a991d397-ea0f-49f0-aebb-5b07aec1fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:05.935561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a25133-c8ba-437d-8b3b-027d03a35f2d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:05.973127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b27590-c106-4f33-9c09-83de118bdd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:05.973755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b27590-c106-4f33-9c09-83de118bdd17 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:05.974032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b27590-c106-4f33-9c09-83de118bdd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:05.974300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b27590-c106-4f33-9c09-83de118bdd17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:05.974945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 160/322] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:05.985881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b27590-c106-4f33-9c09-83de118bdd17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-79b27590-c106-4f33-9c09-83de118bdd17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:05.991153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:05.992067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:05.993508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 163/323] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:06.002884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991d397-ea0f-49f0-aebb-5b07aec1fcb8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.048419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293080c-6e19-49bd-bc06-b0cdeb49a86a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.048712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293080c-6e19-49bd-bc06-b0cdeb49a86a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.048986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293080c-6e19-49bd-bc06-b0cdeb49a86a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.050442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 164/324] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:06.055056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805414d-2200-4f9b-91e5-854accb265f4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.055337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805414d-2200-4f9b-91e5-854accb265f4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.055636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805414d-2200-4f9b-91e5-854accb265f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.073807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d805414d-2200-4f9b-91e5-854accb265f4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:06.075008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d805414d-2200-4f9b-91e5-854accb265f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:06.093344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:06.103698 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991d397-ea0f-49f0-aebb-5b07aec1fcb8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.138900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.139529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.139796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.140118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.140771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:06.141117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.142288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 165/325] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:06.142544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.151281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 161/326] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:06.156666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805414d-2200-4f9b-91e5-854accb265f4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.175062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:06.256864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.266359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805414d-2200-4f9b-91e5-854accb265f4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.301946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.302580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.303157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.303431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] PATH_INFO: `/v3/users/7b03d1d47d934d9eb0b5e10400d3543e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.304725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] RBAC: Authorizing `identity:get_user(user_id=7b03d1d47d934d9eb0b5e10400d3543e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.306167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.308264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 166/327] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:06 2022] GET /identity/v3/users/7b03d1d47d934d9eb0b5e10400d3543e => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:06.313754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.361908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a3a7-e117-4d70-829f-5a589259443d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.397778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:06.398767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.399631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.400621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.401418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.411933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:06.413702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10bec4-825a-4bf9-8315-b979c693196d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:06.417367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:06.418259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:06.419039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 162/328] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:06.424291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.440944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:06.449173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.449808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.450086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.450383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.451094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] RBAC: Adding query filter params (role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, user.id=7b03d1d47d934d9eb0b5e10400d3543e, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:06.451399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.452874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.464279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 167/329] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 22:49:06 2022] GET /identity/v3/role_assignments?user.id=7b03d1d47d934d9eb0b5e10400d3543e&scope.system=all&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 510 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:06.464279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.544734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a9e3d-870b-4d80-bbf2-61c973217c70 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.579255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208726ef-20b0-4719-aac4-b0bd432d0533 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.591639 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.592572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.592997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.593572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.595008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.596780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.599151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 163/330] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:06.605216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.620293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.621192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.621613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.622031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.622714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:06.623015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.624505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.634258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 168/331] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:06.639939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.651187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:06.706132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:06.728361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff2f5d-ed4d-46da-a440-5550d9ab0890 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.754599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea8e9-2745-4233-a0ce-b63d3dafe8b3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.761440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.762048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.762328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.762595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] PATH_INFO: `/v3/groups` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.763221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.764691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.782021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 164/332] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Sep 9 22:49:06 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:49:06.791325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0a7c-3aab-409b-a79b-3126c8e14e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.792036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0a7c-3aab-409b-a79b-3126c8e14e3e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.792324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0a7c-3aab-409b-a79b-3126c8e14e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.792653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0a7c-3aab-409b-a79b-3126c8e14e3e None admin] PATH_INFO: `/v3/users/1dca1c18cf484f879abde1bc588a2e06` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.793917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0a7c-3aab-409b-a79b-3126c8e14e3e None admin] RBAC: Authorizing `identity:get_user(user_id=1dca1c18cf484f879abde1bc588a2e06)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.795787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0a7c-3aab-409b-a79b-3126c8e14e3e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.797471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 169/333] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:06 2022] GET /identity/v3/users/1dca1c18cf484f879abde1bc588a2e06 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:06.803853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:06.916045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413a69-028b-480b-9d49-bb4683fcf5bf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:06.952794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196d293-c553-473d-9fa6-2264ef37e80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:06.953468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196d293-c553-473d-9fa6-2264ef37e80e None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:06.953749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196d293-c553-473d-9fa6-2264ef37e80e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:06.954022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196d293-c553-473d-9fa6-2264ef37e80e None admin] PATH_INFO: `/v3/system/users/1dca1c18cf484f879abde1bc588a2e06/roles/78d8f7f202354798acd7876e3bf7deff` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:06.955519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196d293-c553-473d-9fa6-2264ef37e80e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1dca1c18cf484f879abde1bc588a2e06, role_id=78d8f7f202354798acd7876e3bf7deff)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:06.956916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196d293-c553-473d-9fa6-2264ef37e80e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:06.966507 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 165/334] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Sep 9 22:49:06 2022] PUT /identity/v3/system/users/1dca1c18cf484f879abde1bc588a2e06/roles/78d8f7f202354798acd7876e3bf7deff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:09.309282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f674c-6c76-49e6-aecc-a04c2bd4347b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:09.309879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f674c-6c76-49e6-aecc-a04c2bd4347b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:09.309879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f674c-6c76-49e6-aecc-a04c2bd4347b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:09.311661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 170/335] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:09.319192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ae189-652e-4713-8d57-e72d4ae38f9b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:09.319712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ae189-652e-4713-8d57-e72d4ae38f9b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:09.319786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ae189-652e-4713-8d57-e72d4ae38f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:09.333951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:09.345702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1b5ae189-652e-4713-8d57-e72d4ae38f9b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:09.346898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1b5ae189-652e-4713-8d57-e72d4ae38f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:09.442449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0535a5-7a23-4a45-89a4-c4b2cfebb174 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:09.442773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0535a5-7a23-4a45-89a4-c4b2cfebb174 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:09.442985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0535a5-7a23-4a45-89a4-c4b2cfebb174 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:09.444335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 171/336] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:09.445916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 166/337] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:09.451329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9899f-f00a-4729-9383-6f9f7a6d859a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:09.451642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9899f-f00a-4729-9383-6f9f7a6d859a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:09.451915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9899f-f00a-4729-9383-6f9f7a6d859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:09.492295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5fc9899f-f00a-4729-9383-6f9f7a6d859a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:09.494394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5fc9899f-f00a-4729-9383-6f9f7a6d859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:09.584666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 172/338] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:09.756496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b7f6-6302-4ec5-912a-b96b3d705a6a None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:09.756827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b7f6-6302-4ec5-912a-b96b3d705a6a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:09.757119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b7f6-6302-4ec5-912a-b96b3d705a6a None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:09.758504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 167/339] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:09.765821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197d393-bbdb-4a3e-9fb3-dd9654fa5ae0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:09.766106 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197d393-bbdb-4a3e-9fb3-dd9654fa5ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:09.766380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197d393-bbdb-4a3e-9fb3-dd9654fa5ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:09.776571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:09.789101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4197d393-bbdb-4a3e-9fb3-dd9654fa5ae0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:09.790303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4197d393-bbdb-4a3e-9fb3-dd9654fa5ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:09.901897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 173/340] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:09.963273 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b7f6-6302-4ec5-912a-b96b3d705a6a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.053875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197d393-bbdb-4a3e-9fb3-dd9654fa5ae0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.104798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b7f6-6302-4ec5-912a-b96b3d705a6a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:10.174127 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:10.175092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.175595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.176082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.182519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:10.201357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:10.206547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:10.207415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 22:49:10.208498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:10.213272 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 168/341] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:10.220158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197d393-bbdb-4a3e-9fb3-dd9654fa5ae0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:10.221258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.340387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:10.340913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.341115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.341540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.356289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4e1ff-ee47-43dc-a85d-df0e25c20723 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:10.359853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:10.389744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:10.395116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:10.396150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:10.396953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:10.403678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:10.403678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.403678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.403678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.404041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:10.404041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:10.404041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:10.406295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 174/342] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:49:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:10.416994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 169/343] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:10.418926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.422919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.513979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:10.561185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57935a-b694-46f1-88d7-c32d523ee24b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:10.599729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:10.600634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.601059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.601473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] PATH_INFO: `/v3/users/1dca1c18cf484f879abde1bc588a2e06` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.602821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] RBAC: Authorizing `identity:get_user(user_id=1dca1c18cf484f879abde1bc588a2e06)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:10.605032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:10.607080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 170/344] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:49:10 2022] GET /identity/v3/users/1dca1c18cf484f879abde1bc588a2e06 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:10.608027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.614001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83558-22e4-477d-bda3-3f9a04f8f87a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:10.658693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee49cad-f545-4af2-bc0b-489a69bbfbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:10.659309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee49cad-f545-4af2-bc0b-489a69bbfbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.659612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee49cad-f545-4af2-bc0b-489a69bbfbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.659882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee49cad-f545-4af2-bc0b-489a69bbfbe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.660549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee49cad-f545-4af2-bc0b-489a69bbfbe7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:10.660848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee49cad-f545-4af2-bc0b-489a69bbfbe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:10.662213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee49cad-f545-4af2-bc0b-489a69bbfbe7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:10.675085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 175/345] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:10.676892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd90d8-be81-4ecf-aed5-c717301b1236 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.677183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd90d8-be81-4ecf-aed5-c717301b1236 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.677456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd90d8-be81-4ecf-aed5-c717301b1236 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.678812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 176/346] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:10.679424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd90d8-be81-4ecf-aed5-c717301b1236 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.755953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:10.763368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132e880-2fdc-4091-abab-c1ae32245d80 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:10.799277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:10.810193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:10.810832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.811114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.811518 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.816480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd90d8-be81-4ecf-aed5-c717301b1236 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:10.826379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:10.832249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:10.833220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 171/347] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:10.834556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:10.867611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091448b-032b-4b51-a6c8-aaac220921ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:10.868286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091448b-032b-4b51-a6c8-aaac220921ff None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.869130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091448b-032b-4b51-a6c8-aaac220921ff None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.869398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091448b-032b-4b51-a6c8-aaac220921ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.870176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091448b-032b-4b51-a6c8-aaac220921ff None admin] RBAC: Adding query filter params (role.id=78d8f7f202354798acd7876e3bf7deff, user.id=1dca1c18cf484f879abde1bc588a2e06, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:10.870485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091448b-032b-4b51-a6c8-aaac220921ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:10.872026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091448b-032b-4b51-a6c8-aaac220921ff None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:10.889233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 177/348] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 22:49:10 2022] GET /identity/v3/role_assignments?user.id=1dca1c18cf484f879abde1bc588a2e06&scope.system=all&role.id=78d8f7f202354798acd7876e3bf7deff => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:10.890595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde074f-2e87-4abf-93c8-6ddfcfdc4391 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:10.890866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde074f-2e87-4abf-93c8-6ddfcfdc4391 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:10.891132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde074f-2e87-4abf-93c8-6ddfcfdc4391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:10.916476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ecde074f-2e87-4abf-93c8-6ddfcfdc4391 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:10.917999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ecde074f-2e87-4abf-93c8-6ddfcfdc4391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:10.965859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e345b-a746-4912-948b-8c659ea97220 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:11.007355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 178/349] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:11.008169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:11.008295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde074f-2e87-4abf-93c8-6ddfcfdc4391 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.009130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.009747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.010242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.065452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.078661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.094810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:11.096866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:11.154388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde074f-2e87-4abf-93c8-6ddfcfdc4391 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:11.208313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:11.208998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.209211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.209480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.210154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:11.210453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:11.211868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:11.213069 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 172/350] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:11.219109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.226200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 179/351] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:11.236589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.333155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af605d77-4a8d-4abd-8860-71e7a1d2344c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:11.350825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.377542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e741207-1b23-413e-9671-1a03691664d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:11.378169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e741207-1b23-413e-9671-1a03691664d1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.378448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e741207-1b23-413e-9671-1a03691664d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.378956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e741207-1b23-413e-9671-1a03691664d1 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.381530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077de0a-d2a0-41b2-9e3d-2ee181dab54c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:11.383617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e741207-1b23-413e-9671-1a03691664d1 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:11.385620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e741207-1b23-413e-9671-1a03691664d1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:11.387481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 173/352] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:11.395558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.397538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d981c2-4308-4956-b8ba-950c1a0f66f3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.397820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d981c2-4308-4956-b8ba-950c1a0f66f3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.398470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d981c2-4308-4956-b8ba-950c1a0f66f3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.400476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 174/353] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:11.403742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d981c2-4308-4956-b8ba-950c1a0f66f3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.429652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:11.430269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.430544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.430816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.502741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:11.504248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:11.529321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d981c2-4308-4956-b8ba-950c1a0f66f3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:11.571156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58246f6e-c70d-487b-8e2a-c69de9bb7a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:11.572008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58246f6e-c70d-487b-8e2a-c69de9bb7a19 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.572377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58246f6e-c70d-487b-8e2a-c69de9bb7a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.572723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58246f6e-c70d-487b-8e2a-c69de9bb7a19 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.581614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58246f6e-c70d-487b-8e2a-c69de9bb7a19 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:11.583251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58246f6e-c70d-487b-8e2a-c69de9bb7a19 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:11.587894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 175/354] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:11.587894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603f4c-3e2f-4fbe-b201-a0aeb9277e4b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.587894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603f4c-3e2f-4fbe-b201-a0aeb9277e4b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.587894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603f4c-3e2f-4fbe-b201-a0aeb9277e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.609629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f6603f4c-3e2f-4fbe-b201-a0aeb9277e4b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:11.610857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f6603f4c-3e2f-4fbe-b201-a0aeb9277e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:11.624436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 180/355] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:11.625177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.628949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.634715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.709803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 176/356] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:11.714173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603f4c-3e2f-4fbe-b201-a0aeb9277e4b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.751321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca11000-b8f8-4ad2-91ee-b0b12f85ff7c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:11.789595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:11.790234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.790716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.790995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.807276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:11.812219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:11.813118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:11.813893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 181/357] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:11.814547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.860283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603f4c-3e2f-4fbe-b201-a0aeb9277e4b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:11.892229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.902477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08103453-4bd8-4472-a99c-f7ac8d41c898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:11.903345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08103453-4bd8-4472-a99c-f7ac8d41c898 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.903849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08103453-4bd8-4472-a99c-f7ac8d41c898 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.904298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08103453-4bd8-4472-a99c-f7ac8d41c898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.905187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08103453-4bd8-4472-a99c-f7ac8d41c898 None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=cda287745a6b42a3945fe2fed58f8120) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:11.905662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08103453-4bd8-4472-a99c-f7ac8d41c898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:11.907271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08103453-4bd8-4472-a99c-f7ac8d41c898 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:11.913061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 177/358] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:11.915398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba9be1-ad9e-42dd-9ee1-a968781c74fd None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:11.915974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba9be1-ad9e-42dd-9ee1-a968781c74fd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:11.916422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba9be1-ad9e-42dd-9ee1-a968781c74fd None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:11.918224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 178/359] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:11.919177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba9be1-ad9e-42dd-9ee1-a968781c74fd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:11.937672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:11.955738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b00ba2-cadf-4adb-80e5-a1ec7bbd4bad None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.010184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37289960-f5a6-48fb-bd22-f9a9926a3b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.011224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37289960-f5a6-48fb-bd22-f9a9926a3b80 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.011707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37289960-f5a6-48fb-bd22-f9a9926a3b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.012303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37289960-f5a6-48fb-bd22-f9a9926a3b80 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.013729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37289960-f5a6-48fb-bd22-f9a9926a3b80 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:12.016147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37289960-f5a6-48fb-bd22-f9a9926a3b80 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:12.019251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 182/360] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:12.021102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48138a64-c79c-45a5-890e-712e279817bb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.021643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48138a64-c79c-45a5-890e-712e279817bb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.022357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48138a64-c79c-45a5-890e-712e279817bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.047524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-48138a64-c79c-45a5-890e-712e279817bb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:12.048795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-48138a64-c79c-45a5-890e-712e279817bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:12.052760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba9be1-ad9e-42dd-9ee1-a968781c74fd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.098024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.098648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.098932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.099207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.099977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:12.100246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:12.101847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:12.114483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 179/361] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:12.116358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.139849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 183/362] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:12.143046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48138a64-c79c-45a5-890e-712e279817bb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.209528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:12.225988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:12.241768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac7d72-bb85-4cb6-b334-9c9e3ba7c936 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.280043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48138a64-c79c-45a5-890e-712e279817bb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.282690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.283326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.283685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.283955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.285071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:12.286503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:12.288202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 180/363] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:12.288969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.335864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.336511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.336790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.337062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.356685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:12.362527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:12.363510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:12.364372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:12.365058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 184/364] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:12.365714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.413062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76295386-e560-43ff-b6ab-ec872874bb21 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.456269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.457159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.457593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.458014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.471970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:12.507890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e808-d7a2-4c2d-bcc2-36d7fa696acc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.536579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:12.543191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:12.544709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:12.551885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.552735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.553202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.554060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.554942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=cda287745a6b42a3945fe2fed58f8120) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:12.555440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:12.558026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:12.567326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 185/365] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:12.570136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.677933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 181/366] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:12.678572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.715009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a2cec-e2fa-4201-a635-d15c6073647b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.740470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:12.752124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.752796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.753074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.753392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.754113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:12.754421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:12.755827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:12.765972 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 186/367] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:12.766852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21b5f7-3a55-440b-a5c9-686749c93442 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.822701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:12.850028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.850676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.850960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.851242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.862788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:12.867736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:12.868686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:12.869478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 182/368] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:12.870138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.881569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbc506-aacb-4818-83eb-c72a898192a4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.919202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:12.920200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:12.920518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:12.920794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:12.923012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:12.924485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:12.926509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 187/369] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:12.927344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:12.995667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d707b3-9655-42e1-ac62-2840dd1e3d1c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:12.997316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.036169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.036937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.037222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.037495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.039608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab299-8229-412d-9339-cfb7952005f0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.057448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.075001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.075730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.076079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.076423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.077153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:13.077524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.078962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.091623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 188/370] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:13.092569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.116067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.117647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.210180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905afd0-a752-4c3d-b3aa-da19b0b59acd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.248192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 183/371] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:13.248810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.253449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.266857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.267647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.267931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.268207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.269353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.271028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.274166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 189/372] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:13.275127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.336451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.356133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776642a-22ca-4641-9a78-94cffdbbabf2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.392520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.393120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.393399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.393672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.401827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46908f93-7a97-4d90-a4bb-e3d16e3a7dc2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.446065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.446299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.447107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.447430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.449263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.451361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.454023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 190/373] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:13.454825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.497583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.498086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.500590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.580062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9c17-d2a9-4ef8-a6af-7fcdef1305d1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.598713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.619681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.620764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.621320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.621846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.622761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=eb87cf3a34c846e8baad667e5145e480) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:13.623283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.624335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 184/374] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:13.625181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.625306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.631965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 191/375] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:13.633070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.757868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c240d4-2c52-4b61-a6a5-aef5c13ddabb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.760414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.776944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf80842-4a4b-4971-8bff-1d8f7b7992f0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.803163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.803163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.803455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.803710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.805305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.807706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.809648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 185/376] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:13.816396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.821490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.822988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.823481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.823979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] PATH_INFO: `/v3/projects/8675cee43cc84599b492f8c81300cc82` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.825413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] RBAC: Authorizing `identity:get_project(project_id=8675cee43cc84599b492f8c81300cc82)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.827159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.830011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 192/377] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/projects/8675cee43cc84599b492f8c81300cc82 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:13.835964 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:13.892960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:13.940717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a5b1a-cb76-48c8-a5ef-b8bdb06786f5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.964504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a70b16-366a-4f8e-bb47-678dbadb0982 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:13.985119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:13.985746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:13.986035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:13.986305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:13.987393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=eb87cf3a34c846e8baad667e5145e480) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:13.987756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:13.989316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:13.998094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 186/378] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:14.013643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b770ba-a3c3-4f84-9ab8-a05c65115985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:14.014758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b770ba-a3c3-4f84-9ab8-a05c65115985 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:14.015262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b770ba-a3c3-4f84-9ab8-a05c65115985 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:14.015799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b770ba-a3c3-4f84-9ab8-a05c65115985 None admin] PATH_INFO: `/v3/groups/3d0dec5c7c914359aec3cfbd78037779` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:14.023277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b770ba-a3c3-4f84-9ab8-a05c65115985 None admin] RBAC: Authorizing `identity:get_group(group_id=3d0dec5c7c914359aec3cfbd78037779)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:14.025774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b770ba-a3c3-4f84-9ab8-a05c65115985 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:14.028095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 193/379] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:13 2022] GET /identity/v3/groups/3d0dec5c7c914359aec3cfbd78037779 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:14.034308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:14.086587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:14.178638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13545c-5f3a-46d3-9ac5-dee998146312 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:14.223269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003b61d-8361-4ebc-a898-b64951745f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:14.223931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003b61d-8361-4ebc-a898-b64951745f19 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:14.224216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003b61d-8361-4ebc-a898-b64951745f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:14.224491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003b61d-8361-4ebc-a898-b64951745f19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:14.225233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003b61d-8361-4ebc-a898-b64951745f19 None admin] RBAC: Adding query filter params (group.id=3d0dec5c7c914359aec3cfbd78037779, role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=8675cee43cc84599b492f8c81300cc82) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:14.225612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003b61d-8361-4ebc-a898-b64951745f19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:14.227060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003b61d-8361-4ebc-a898-b64951745f19 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:14.234345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 187/380] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:14 2022] GET /identity/v3/role_assignments?group.id=3d0dec5c7c914359aec3cfbd78037779&scope.project.id=8675cee43cc84599b492f8c81300cc82&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:14.454900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c36cc-ff9a-4cc9-bf67-4f0d2db62969 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:14.455237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c36cc-ff9a-4cc9-bf67-4f0d2db62969 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:14.455425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c36cc-ff9a-4cc9-bf67-4f0d2db62969 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:14.456752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 194/381] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:14.461917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79b01-bc3c-44f5-853d-99e51aca8f9c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:14.462309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79b01-bc3c-44f5-853d-99e51aca8f9c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:14.462591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79b01-bc3c-44f5-853d-99e51aca8f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:14.483693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8da79b01-bc3c-44f5-853d-99e51aca8f9c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:14.484896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8da79b01-bc3c-44f5-853d-99e51aca8f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:14.561559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 188/382] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:14.841295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c36cc-ff9a-4cc9-bf67-4f0d2db62969 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:14.979069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c36cc-ff9a-4cc9-bf67-4f0d2db62969 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:14.992821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:15.015350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:15.016066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:15.016342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:15.016610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:15.032331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:15.037233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:15.038193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:15.039027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 195/383] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:49:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:15.045339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79b01-bc3c-44f5-853d-99e51aca8f9c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:15.094667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:15.195365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79b01-bc3c-44f5-853d-99e51aca8f9c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:15.247596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bf0e7-87f9-4c4c-9478-de549135604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:15.248129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf0e7-87f9-4c4c-9478-de549135604b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:15.248405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf0e7-87f9-4c4c-9478-de549135604b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:15.248671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf0e7-87f9-4c4c-9478-de549135604b None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:15.249340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bf0e7-87f9-4c4c-9478-de549135604b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:15.249640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bf0e7-87f9-4c4c-9478-de549135604b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:15.251038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bf0e7-87f9-4c4c-9478-de549135604b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:15.263960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 189/384] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:15.269733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:15.338196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb628e-341f-41d5-92d1-ecc622df8e08 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:15.338556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb628e-341f-41d5-92d1-ecc622df8e08 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:15.338749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb628e-341f-41d5-92d1-ecc622df8e08 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:15.340350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 190/385] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:15.345513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eb53d-7111-40da-b990-bd33b9432953 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:15.345788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eb53d-7111-40da-b990-bd33b9432953 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:15.346058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eb53d-7111-40da-b990-bd33b9432953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:15.365982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-649eb53d-7111-40da-b990-bd33b9432953 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:15.367164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-649eb53d-7111-40da-b990-bd33b9432953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:15.461720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe9406-abb8-42e2-8d61-2860e5a3bc80 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:15.477750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 191/386] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:15.513545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:15.514205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:15.514483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:15.514813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:15.516426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:15.517918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:15.524054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 196/387] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:15 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:15.530117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649eb53d-7111-40da-b990-bd33b9432953 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:15.561068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:15.671933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649eb53d-7111-40da-b990-bd33b9432953 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:15.721514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:15.722161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:15.722442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:15.722713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:15.723893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:15.725320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:15.727245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 192/388] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:15 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:15.734143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:15.788636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:15.870374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4e80a-94cb-4fa8-9ee6-d89d07740877 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:15.877343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:15.942997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:15.943711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:15.944000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:15.944269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:15.949766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cda287745a6b42a3945fe2fed58f8120, group_id=d34efc87c7e44ac6932e9b44734c1dce, role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:15.951591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:15.976287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 197/389] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 22:49:15 2022] PUT /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:16.019695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:16.036444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a40686-f71c-46a7-8c3c-019c7b1fac2b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:16.084651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:16.085218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:16.085502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:16.085770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:16.103262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:16.108468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:16.109393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:16.110196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 193/390] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:16.115902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:16.260578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:16.264645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37943c6b-e74c-4d48-b53b-f82a6421bad5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:16.311661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:16.312614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:16.313051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:16.313475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:16.314315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:16.314801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:16.316402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:16.326151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 198/391] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:16.331597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:16.455170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab33be-08e9-4f1c-a0ac-a1197c1fd95a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:16.503324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bd870-c032-4c54-9929-d3c0c54f138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:16.503993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bd870-c032-4c54-9929-d3c0c54f138d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:16.504277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bd870-c032-4c54-9929-d3c0c54f138d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:16.504553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bd870-c032-4c54-9929-d3c0c54f138d None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:16.505690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bd870-c032-4c54-9929-d3c0c54f138d None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:16.507132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bd870-c032-4c54-9929-d3c0c54f138d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:16.509592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 194/392] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:16 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:16.515042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:16.590973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5b5dd-1cc4-4aeb-b3a1-ab5101e1fc8c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:16.591292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5b5dd-1cc4-4aeb-b3a1-ab5101e1fc8c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:16.591572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5b5dd-1cc4-4aeb-b3a1-ab5101e1fc8c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:16.593193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 195/393] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:16.599840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f3e3-1b58-46f7-a163-077336c83dbf None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:16.600133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f3e3-1b58-46f7-a163-077336c83dbf None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:16.600417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f3e3-1b58-46f7-a163-077336c83dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:16.635634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0f86f3e3-1b58-46f7-a163-077336c83dbf None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:16.636848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0f86f3e3-1b58-46f7-a163-077336c83dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:16.664762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:16.690153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c69b24-d4e7-40e7-a589-5cfca6be014e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:16.766257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:16.766904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:16.767186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:16.767459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:16.770729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:16.772217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:16.773879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 199/394] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:16 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:16.781761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 196/395] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:16.782345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:16.818530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:16.910359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01fa46-5c69-46bf-8a10-ea319c854a5c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:16.964014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb21c7-473a-4c3c-a33a-c8b00274836b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:16.965031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb21c7-473a-4c3c-a33a-c8b00274836b None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:16.965450 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb21c7-473a-4c3c-a33a-c8b00274836b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:16.965868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb21c7-473a-4c3c-a33a-c8b00274836b None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:16.968055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb21c7-473a-4c3c-a33a-c8b00274836b None admin] RBAC: Authorizing `identity:create_grant(project_id=cda287745a6b42a3945fe2fed58f8120, group_id=d34efc87c7e44ac6932e9b44734c1dce, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:16.969654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb21c7-473a-4c3c-a33a-c8b00274836b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:16.988381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 200/396] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 22:49:16 2022] PUT /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:17.057322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70994259-a27f-4fd3-a659-dfe73f0556dd None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.057613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70994259-a27f-4fd3-a659-dfe73f0556dd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.057893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70994259-a27f-4fd3-a659-dfe73f0556dd None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.059404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 197/397] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:17.065741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bdb83-d4d8-4d7d-aef8-1c59751a38c2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.066021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bdb83-d4d8-4d7d-aef8-1c59751a38c2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.066299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bdb83-d4d8-4d7d-aef8-1c59751a38c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.086771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-432bdb83-d4d8-4d7d-aef8-1c59751a38c2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:17.088553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-432bdb83-d4d8-4d7d-aef8-1c59751a38c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:17.195890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 201/398] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:17.242879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70994259-a27f-4fd3-a659-dfe73f0556dd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:17.363535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70994259-a27f-4fd3-a659-dfe73f0556dd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:17.393702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab58a65-9786-4bff-af4d-818b622ec9af None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.394007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab58a65-9786-4bff-af4d-818b622ec9af None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.394235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab58a65-9786-4bff-af4d-818b622ec9af None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.395745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 202/399] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:17.401157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d8f9-ccbb-4618-9892-cc8b44ab18de None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.401666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d8f9-ccbb-4618-9892-cc8b44ab18de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.402145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d8f9-ccbb-4618-9892-cc8b44ab18de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.403299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:17.403964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.404236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.404501 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.422817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2c56d8f9-ccbb-4618-9892-cc8b44ab18de None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:17.424419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2c56d8f9-ccbb-4618-9892-cc8b44ab18de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:17.425136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:17.431540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:17.432485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:17.433270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:17.439641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 198/400] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:17.442743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:17.455117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:17.456985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:17.534579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 203/401] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:17.553767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56d8f9-ccbb-4618-9892-cc8b44ab18de None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:17.604175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd8d12-d8ac-48ad-9942-a9eae600641d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:17.655837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:17.657787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.658068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.658337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.658999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] RBAC: Adding query filter params (name=member) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:17.659298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:17.660696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:17.682396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 199/402] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:17.687350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:17.715903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56d8f9-ccbb-4618-9892-cc8b44ab18de None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:17.775869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:17.776511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.776786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.777052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.792873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:17.799800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:17.804949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:17.806388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:17.807182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 204/403] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:49:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:17.820678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:17.845012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd0637-70fb-4d63-a6a7-582d39a96198 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:17.847410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:17.891251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:17.892357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:17.893586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:17.894642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:17.896127 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:17.899028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:17.901618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 200/404] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:17 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:17.902551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:17.973402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3b829-671d-4a2f-b728-6d16bd5f685c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.009414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.009940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689be5a-7c58-42f3-bef5-d329dfc260d5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.010072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.010304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.010576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.011238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:18.011561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.012952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.025888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 205/405] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:18.025888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.052357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.053206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.053653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.054072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.057165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:18.067438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:18.073459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:18.074528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:18.075384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:18.077462 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 201/406] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:18.078366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.100464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:18.132051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735ab9-9b97-4726-af3d-f265530fcff0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.167279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.167951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.168231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.168502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.169635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.171338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.173314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 206/407] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:18 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:18.174169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.185495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d00a4-50f6-483c-802f-d614495766dd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.224165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.225065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.225517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.225941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.227240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.229417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.235747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 202/408] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:18 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:18.236720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.281874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeedb83-30c0-4258-92a3-f3c79a0f3715 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.305396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:18.317149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.317768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.318052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.318328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.318998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:18.319303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.320716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.334613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 207/409] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:18.335529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.338202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:18.351096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff648f4-4be4-4b6e-a6a5-26df79458280 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.387132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.388104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.388553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.388975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.391144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] RBAC: Authorizing `identity:create_grant(project_id=eb87cf3a34c846e8baad667e5145e480, group_id=d34efc87c7e44ac6932e9b44734c1dce, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.392812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.410067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 203/410] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 22:49:18 2022] PUT /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:18.411160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.443713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab5fd-780a-4a0d-88f6-b52f3c678da6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.481232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.482043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.482189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.482426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.483599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.485047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.486975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 208/411] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:18 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:18.493184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.589853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9421cc9-cd3a-4826-967d-082af451252f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.620374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:18.622169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:18.636408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.637319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.637634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.637956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] PATH_INFO: `/v3/groups/3d0dec5c7c914359aec3cfbd78037779` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.639347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] RBAC: Authorizing `identity:get_group(group_id=3d0dec5c7c914359aec3cfbd78037779)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.641301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.644118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 204/412] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:18 2022] GET /identity/v3/groups/3d0dec5c7c914359aec3cfbd78037779 => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:18.649220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.679997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba999a79-e489-49f4-8920-7573f549ec8c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.731346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c7eec-ce42-4323-bc05-29c7e37ab098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.734584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c7eec-ce42-4323-bc05-29c7e37ab098 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.734584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c7eec-ce42-4323-bc05-29c7e37ab098 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.734584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c7eec-ce42-4323-bc05-29c7e37ab098 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.735608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c7eec-ce42-4323-bc05-29c7e37ab098 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb87cf3a34c846e8baad667e5145e480, group_id=d34efc87c7e44ac6932e9b44734c1dce, role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.737173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c7eec-ce42-4323-bc05-29c7e37ab098 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.756184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 209/413] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 22:49:18 2022] PUT /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480/groups/d34efc87c7e44ac6932e9b44734c1dce/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:18.783240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43673e64-c9a6-498e-9543-2522b9363b6a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:18.822532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:18.823355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb38bbc-2a77-40b8-805a-8fed6c2dfd55 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.823670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb38bbc-2a77-40b8-805a-8fed6c2dfd55 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.823943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb38bbc-2a77-40b8-805a-8fed6c2dfd55 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.824424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.824845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.825249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] PATH_INFO: `/v3/projects/8675cee43cc84599b492f8c81300cc82` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.826571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] RBAC: Authorizing `identity:get_project(project_id=8675cee43cc84599b492f8c81300cc82)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:18.828171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:18.829274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 210/414] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:18.830529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 205/415] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:18 2022] GET /identity/v3/projects/8675cee43cc84599b492f8c81300cc82 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:18.830987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757f64-1c66-469c-a795-25861bb68109 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:18.831326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757f64-1c66-469c-a795-25861bb68109 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:18.835096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757f64-1c66-469c-a795-25861bb68109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:18.836367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:18.869530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-90757f64-1c66-469c-a795-25861bb68109 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:18.870647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-90757f64-1c66-469c-a795-25861bb68109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:18.907064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:18.958308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab2a46-a45c-4110-ab12-9f0e4e5a155d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:19.020615 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:19.021281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:19.021562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:19.021834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] PATH_INFO: `/v3/projects/8675cee43cc84599b492f8c81300cc82/groups/3d0dec5c7c914359aec3cfbd78037779/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:19.024336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8675cee43cc84599b492f8c81300cc82, group_id=3d0dec5c7c914359aec3cfbd78037779, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:19.026006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 211/416] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:19.032008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:19.051474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 206/417] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 22:49:18 2022] PUT /identity/v3/projects/8675cee43cc84599b492f8c81300cc82/groups/3d0dec5c7c914359aec3cfbd78037779/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:19.378529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757f64-1c66-469c-a795-25861bb68109 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:19.449916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:19.507526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757f64-1c66-469c-a795-25861bb68109 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:19.557161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec801e-84c3-47e3-9e48-e3a7972c849b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:19.557804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec801e-84c3-47e3-9e48-e3a7972c849b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:19.558076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec801e-84c3-47e3-9e48-e3a7972c849b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:19.558343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec801e-84c3-47e3-9e48-e3a7972c849b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:19.574141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec801e-84c3-47e3-9e48-e3a7972c849b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ffec801e-84c3-47e3-9e48-e3a7972c849b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:19.579088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:19.580058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:19.580842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:19.583023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 212/418] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:49:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:19.588317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:19.706752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99227d1-7ba4-44c9-9c83-7ba6085fa136 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:19.707074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99227d1-7ba4-44c9-9c83-7ba6085fa136 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:19.707253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99227d1-7ba4-44c9-9c83-7ba6085fa136 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:19.709158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 213/419] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:19.717369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5c7d-5f87-46f5-9edb-6a5126e3aeb4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:19.718056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdb356-216c-4213-ae7e-9ef2502313cd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:19.718348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdb356-216c-4213-ae7e-9ef2502313cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:19.718624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdb356-216c-4213-ae7e-9ef2502313cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:19.745481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2bfdb356-216c-4213-ae7e-9ef2502313cd None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:19.746755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2bfdb356-216c-4213-ae7e-9ef2502313cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:19.747623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:19.777065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:19.777683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:19.777968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:19.778244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:19.778912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:19.779215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:19.780630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:19.804477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 207/420] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:19.810508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:19.842157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 214/421] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:19.963563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a3a4e-653f-4679-b1f7-3998c0838f2e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:20.015045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:20.015689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:20.015971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:20.016241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:20.081933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:20.106873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:20.108388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:20.115165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdb356-216c-4213-ae7e-9ef2502313cd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:20.209087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 208/422] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:20.209718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:20.215964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:20.303045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdb356-216c-4213-ae7e-9ef2502313cd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:20.370012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:20.371273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:20.372472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:20.373288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:20.374517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:20.394804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:20.401694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:20.402552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:20.403354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:20.407929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 215/423] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:20.410167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb9d7e-8733-40e1-a23e-5e4066f33346 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:20.413608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:20.473183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:20.474135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:20.475413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:20.476738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:20.478799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:20.483999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:20.485915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 209/424] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:20 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:20.491388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:20.598201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6d925-09eb-4696-a0eb-0bdd8d36ed52 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:20.617502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:20.662947 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11121b72-05e6-449d-ac16-b6074bb4172d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:20.673999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:20.674624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:20.674906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:20.675177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:20.675891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:20.676193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:20.677579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:20.691332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 216/425] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:20.698445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:20.741371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:20.742006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:20.742297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:20.742577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:20.744857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:20.746564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:20.755336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 210/426] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:20 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:20.757100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:20.800544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:20.869666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367fb43-a845-4836-9320-6ad0238c70f8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:20.936880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:20.937517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:20.937792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:20.938059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:20.957781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60e41-430d-4224-989e-83a09c2f3570 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:21.019618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:21.025453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa1af9-1879-43b5-bd04-7a4cc03f1092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:21.026302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa1af9-1879-43b5-bd04-7a4cc03f1092 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.026713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa1af9-1879-43b5-bd04-7a4cc03f1092 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.027116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa1af9-1879-43b5-bd04-7a4cc03f1092 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.028007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa1af9-1879-43b5-bd04-7a4cc03f1092 None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=cda287745a6b42a3945fe2fed58f8120) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:21.028314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa1af9-1879-43b5-bd04-7a4cc03f1092 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:21.029740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa1af9-1879-43b5-bd04-7a4cc03f1092 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:21.044293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 211/427] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:20 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 611 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:21.060663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:21.062304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:21.065397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cac33-4c26-4219-9e35-b19e1c20d7ec None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.065750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cac33-4c26-4219-9e35-b19e1c20d7ec None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.066035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cac33-4c26-4219-9e35-b19e1c20d7ec None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.067595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 212/428] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:21.073636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb675a49-1625-4f26-bbdf-eb53d96950e7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.073944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb675a49-1625-4f26-bbdf-eb53d96950e7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.074184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb675a49-1625-4f26-bbdf-eb53d96950e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.093679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fb675a49-1625-4f26-bbdf-eb53d96950e7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:21.094939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fb675a49-1625-4f26-bbdf-eb53d96950e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:21.233206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 217/429] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 22:49:21.237162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:21.240046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 213/430] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:21.320185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:21.356619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225882-c683-41f7-94a6-4aae9579b926 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:21.393847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997c04f-f94b-4a34-860c-87cec95c52a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:21.394497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997c04f-f94b-4a34-860c-87cec95c52a6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.394776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997c04f-f94b-4a34-860c-87cec95c52a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.395050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997c04f-f94b-4a34-860c-87cec95c52a6 None admin] PATH_INFO: `/v3/projects/cda287745a6b42a3945fe2fed58f8120` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.396213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997c04f-f94b-4a34-860c-87cec95c52a6 None admin] RBAC: Authorizing `identity:get_project(project_id=cda287745a6b42a3945fe2fed58f8120)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:21.397676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997c04f-f94b-4a34-860c-87cec95c52a6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:21.400135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 218/431] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:21 2022] GET /identity/v3/projects/cda287745a6b42a3945fe2fed58f8120 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:21.405644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb675a49-1625-4f26-bbdf-eb53d96950e7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:21.421421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:21.459054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4352-a903-489d-9d4b-b985ae531f1c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.459339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4352-a903-489d-9d4b-b985ae531f1c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.459640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4352-a903-489d-9d4b-b985ae531f1c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.461269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 219/432] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:21.466826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d9cf0-ba6d-4363-89c9-ae5cbbb5b8b8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.467115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d9cf0-ba6d-4363-89c9-ae5cbbb5b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.467389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d9cf0-ba6d-4363-89c9-ae5cbbb5b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.486071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7d3d9cf0-ba6d-4363-89c9-ae5cbbb5b8b8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:21.487271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7d3d9cf0-ba6d-4363-89c9-ae5cbbb5b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:21.519516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb675a49-1625-4f26-bbdf-eb53d96950e7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:21.569098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 220/433] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:21.569992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d9cf0-ba6d-4363-89c9-ae5cbbb5b8b8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:21.571190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:21.578470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.578757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.579031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.580362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:21.581805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:21.583688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 214/434] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:21 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:21.595460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:21.664078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:21.703862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:21.704900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d9cf0-ba6d-4363-89c9-ae5cbbb5b8b8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:21.738848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c2837-5eba-4572-bcd5-d2726a0e504b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:21.751094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:21.751771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.752038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.752310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.766692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:21.771693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:21.772588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 22:49:21.773385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:21.777754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 221/435] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:21.778500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:21.786324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379594ea-6f0a-4ddb-a133-2fa6b962500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:21.787189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379594ea-6f0a-4ddb-a133-2fa6b962500b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.787746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379594ea-6f0a-4ddb-a133-2fa6b962500b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.788718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379594ea-6f0a-4ddb-a133-2fa6b962500b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.789683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379594ea-6f0a-4ddb-a133-2fa6b962500b None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=cda287745a6b42a3945fe2fed58f8120) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:21.790756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379594ea-6f0a-4ddb-a133-2fa6b962500b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:21.792829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379594ea-6f0a-4ddb-a133-2fa6b962500b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:21.802058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 215/436] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:21 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=cda287745a6b42a3945fe2fed58f8120&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:21.843073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ee42-4174-4dca-89e3-738cf253a2ad None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.843245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ee42-4174-4dca-89e3-738cf253a2ad None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.843565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ee42-4174-4dca-89e3-738cf253a2ad None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.849063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 216/437] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:21.851021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce69f-3975-4849-b10d-ed3cecfde1ce None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.851230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce69f-3975-4849-b10d-ed3cecfde1ce None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.851482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce69f-3975-4849-b10d-ed3cecfde1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.871636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-402ce69f-3975-4849-b10d-ed3cecfde1ce None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:21.872857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-402ce69f-3975-4849-b10d-ed3cecfde1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:21.903966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a37e3-7e1b-46ba-b678-2119ab7c097e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:21.939193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:21.947080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:21.947751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:21.948024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:21.948291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:21.948970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:21.949272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:21.950674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:21.954053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 217/438] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:21.957107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ce69f-3975-4849-b10d-ed3cecfde1ce None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:21.960515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 222/439] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:49:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:21.969487 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.054027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.093042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ce69f-3975-4849-b10d-ed3cecfde1ce None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.117583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00a3f3-b33a-48a8-b608-215dea39ca76 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.146112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.146761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.147060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.147334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.161134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.161748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.162025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.162294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.164213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:22.171077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:22.172046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:22.172834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:22.173442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 218/440] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:49:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:22.178709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.236163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.248663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:22.250118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:22.300889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e8292-3555-43e6-835a-b4eb171b1973 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.341696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.342636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.343117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.343660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.344531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:22.345036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:22.346674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:22.348980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.358223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 219/441] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:22.359050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.367046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 223/442] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 22:49:22.368042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.470667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.473565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a55bfb-c9b6-45be-b473-1afad64b1933 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.479724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ffc2e-9f0b-49aa-ae73-fe77b7e53fde None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.506182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.506800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.507082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.507380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.512873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960470d-0536-429b-bbac-146312c3221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.513478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960470d-0536-429b-bbac-146312c3221c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.513756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960470d-0536-429b-bbac-146312c3221c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.514026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960470d-0536-429b-bbac-146312c3221c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.517586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:22.523710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:22.525190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:22.525991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 220/443] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:22.526664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.582688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960470d-0536-429b-bbac-146312c3221c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:22.583023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.584193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960470d-0536-429b-bbac-146312c3221c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:22.628263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d5946-8793-4978-a6b9-28f579f43b22 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.661080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.661705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.661986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.662261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.663384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:22.664842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:22.667268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 221/444] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:22.668062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.672162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.681191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 224/445] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:22.681994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960470d-0536-429b-bbac-146312c3221c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.774260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60902-f4ba-4941-b058-eff18f23cdfe None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.783174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960470d-0536-429b-bbac-146312c3221c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.796690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.807900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.808502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.808786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.809059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.809718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:22.810022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:22.811384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:22.817163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.817790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.818065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.818331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.819275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 222/446] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:22.819426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:22.820084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.820976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:22.823062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 225/447] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:22.825478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:22.883140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:22.922887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6888e1b-e702-4b3c-9fc3-bc7d73ddef8c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.923073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998846c-2bcb-499c-9089-a2ea4fec18d4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:22.956102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.956287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:22.956720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.956814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:22.957001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.957090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:22.957274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] PATH_INFO: `/v3/projects/eb87cf3a34c846e8baad667e5145e480` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.957360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:22.958381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] RBAC: Authorizing `identity:get_project(project_id=eb87cf3a34c846e8baad667e5145e480)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:22.959829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:22.961636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 223/448] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/projects/eb87cf3a34c846e8baad667e5145e480 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:22.962207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:23.015686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:23.026095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:23.027588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:23.059986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e847c53-0780-4647-af3a-8cef92251013 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:23.083933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:23.093606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:23.094212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:23.094494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:23.094764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:23.095500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=01c59f6e5c2b4b7a8eac21824e49fb9d, scope.project.id=eb87cf3a34c846e8baad667e5145e480) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:23.095817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:23.097238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:23.103902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 224/449] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=01c59f6e5c2b4b7a8eac21824e49fb9d => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:23.104689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:23.140211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 226/450] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 22:49:23.146048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:23.222418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b10f5-7a12-4dfc-a304-7d050bca6cbf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:23.241327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:23.252930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf101a-b048-4578-93dc-07687b06a5e3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:23.262266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:23.262908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:23.263187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:23.263458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] PATH_INFO: `/v3/groups/d34efc87c7e44ac6932e9b44734c1dce` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:23.264672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] RBAC: Authorizing `identity:get_group(group_id=d34efc87c7e44ac6932e9b44734c1dce)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:23.266132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:23.268053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 225/451] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:23 2022] GET /identity/v3/groups/d34efc87c7e44ac6932e9b44734c1dce => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:23.273566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:23.295988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:23.296634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:23.296938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:23.297174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] PATH_INFO: `/v3/projects/8675cee43cc84599b492f8c81300cc82` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:23.298563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] RBAC: Authorizing `identity:get_project(project_id=8675cee43cc84599b492f8c81300cc82)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:23.300580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:23.301681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 227/452] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:49:23 2022] GET /identity/v3/projects/8675cee43cc84599b492f8c81300cc82 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:23.307423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:23.337935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:23.397845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b1929-ae0a-405a-87fa-c142f070493c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:23.422437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d552eb-8706-442a-aeb3-403599753870 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:23.437138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:23.437380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:23.437782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:23.438063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:23.438787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] RBAC: Adding query filter params (group.id=d34efc87c7e44ac6932e9b44734c1dce, role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=eb87cf3a34c846e8baad667e5145e480) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:23.439092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:23.440573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:23.449369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 226/453] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:23 2022] GET /identity/v3/role_assignments?group.id=d34efc87c7e44ac6932e9b44734c1dce&scope.project.id=eb87cf3a34c846e8baad667e5145e480&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:23.461585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25786ad6-a84a-4895-98ac-7025293b0f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:23.462330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25786ad6-a84a-4895-98ac-7025293b0f74 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:23.462608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25786ad6-a84a-4895-98ac-7025293b0f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:23.462876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25786ad6-a84a-4895-98ac-7025293b0f74 None admin] PATH_INFO: `/v3/groups/3d0dec5c7c914359aec3cfbd78037779` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:23.464150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25786ad6-a84a-4895-98ac-7025293b0f74 None admin] RBAC: Authorizing `identity:get_group(group_id=3d0dec5c7c914359aec3cfbd78037779)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:23.465630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25786ad6-a84a-4895-98ac-7025293b0f74 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:23.467896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 228/454] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 22:49:23 2022] GET /identity/v3/groups/3d0dec5c7c914359aec3cfbd78037779 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:23.473726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:23.530739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:23.582786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765c70f-1f16-4f20-b0df-a16b51faa498 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:23.616613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be5a03-7faa-4139-bdc4-0117901b6a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:23.617218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be5a03-7faa-4139-bdc4-0117901b6a9c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:23.617496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be5a03-7faa-4139-bdc4-0117901b6a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:23.617765 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be5a03-7faa-4139-bdc4-0117901b6a9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:23.619022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be5a03-7faa-4139-bdc4-0117901b6a9c None admin] RBAC: Adding query filter params (group.id=3d0dec5c7c914359aec3cfbd78037779, role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=8675cee43cc84599b492f8c81300cc82) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:23.619325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be5a03-7faa-4139-bdc4-0117901b6a9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:23.620770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be5a03-7faa-4139-bdc4-0117901b6a9c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:23.626536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 227/455] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 22:49:23 2022] GET /identity/v3/role_assignments?group.id=3d0dec5c7c914359aec3cfbd78037779&scope.project.id=8675cee43cc84599b492f8c81300cc82&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:26.440257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1cc7b-05e9-47e4-b523-bfa26dcc1fe7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:26.441491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1cc7b-05e9-47e4-b523-bfa26dcc1fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:26.441973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1cc7b-05e9-47e4-b523-bfa26dcc1fe7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:26.443917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 229/456] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:26.450319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f3776-fb3d-4b5c-9eef-89baac65e378 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:26.451156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f3776-fb3d-4b5c-9eef-89baac65e378 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:26.451694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f3776-fb3d-4b5c-9eef-89baac65e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:26.476444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-df4f3776-fb3d-4b5c-9eef-89baac65e378 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:26.478509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-df4f3776-fb3d-4b5c-9eef-89baac65e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:26.522293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a76a12-8c42-4c80-81f6-9ca591b135e6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:26.522928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a76a12-8c42-4c80-81f6-9ca591b135e6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:26.523468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a76a12-8c42-4c80-81f6-9ca591b135e6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:26.525426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 230/457] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:26.546837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bc512-7105-438b-8e3c-17753137c729 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:26.547140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bc512-7105-438b-8e3c-17753137c729 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:26.547386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bc512-7105-438b-8e3c-17753137c729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:26.559129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 228/458] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:26.564359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef46f6-db46-4280-8cec-1aabe4b982b8 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:26.564649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef46f6-db46-4280-8cec-1aabe4b982b8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:26.564930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef46f6-db46-4280-8cec-1aabe4b982b8 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:26.566463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 229/459] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:26.571987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d49ba-824f-4036-86fc-f8403ed96c8b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:26.572276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d49ba-824f-4036-86fc-f8403ed96c8b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:26.572573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d49ba-824f-4036-86fc-f8403ed96c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:26.583207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8c5bc512-7105-438b-8e3c-17753137c729 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:26.584414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c5bc512-7105-438b-8e3c-17753137c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:26.604099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4f3d49ba-824f-4036-86fc-f8403ed96c8b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:26.605280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4f3d49ba-824f-4036-86fc-f8403ed96c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:26.661239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:26.696625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 230/460] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:26.741740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 231/461] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:26.875796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d49ba-824f-4036-86fc-f8403ed96c8b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:26.990110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d49ba-824f-4036-86fc-f8403ed96c8b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.024786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.025422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.025705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.025980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.036585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.038051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:27.054604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:27.055643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:27.056523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:27.056523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:27.056523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:27.056523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 231/462] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:27.068870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.069008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bc512-7105-438b-8e3c-17753137c729 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.107430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:27.123875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:27.220725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bc512-7105-438b-8e3c-17753137c729 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.227937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02027716-d8d0-4c8b-846d-025359b9d06d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.260823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.261533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.261820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.262090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.262735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:27.263037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.264481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.267668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0c90d-66da-4f18-be1f-762594337435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.268299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0c90d-66da-4f18-be1f-762594337435 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.268575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0c90d-66da-4f18-be1f-762594337435 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.268844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0c90d-66da-4f18-be1f-762594337435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.277146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 232/463] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:27.277810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.286292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0c90d-66da-4f18-be1f-762594337435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.287774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0c90d-66da-4f18-be1f-762594337435 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0ef0c90d-66da-4f18-be1f-762594337435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:27.302873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:27.303818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:27.304679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:27.304679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:27.304679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:27.304679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 232/464] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:27.309391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8c422-13f1-41f4-be71-fd95ac6851e4 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.309829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8c422-13f1-41f4-be71-fd95ac6851e4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.310252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8c422-13f1-41f4-be71-fd95ac6851e4 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.311839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 233/465] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:27.312386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8c422-13f1-41f4-be71-fd95ac6851e4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.371688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:27.390083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fdaa6-5db5-424f-88a4-7fdeb2fa788d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.399349 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:27.424321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8c422-13f1-41f4-be71-fd95ac6851e4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.424749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.425404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.425693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.425961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.438036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.439477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:27.453763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:27.455063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:27.455945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:27.455945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:27.455945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:27.459837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 233/466] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:27.460531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.462757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12aff6-3312-42b7-9f94-e62d2c40cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.463370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12aff6-3312-42b7-9f94-e62d2c40cc01 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.463698 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12aff6-3312-42b7-9f94-e62d2c40cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.463962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12aff6-3312-42b7-9f94-e62d2c40cc01 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.464602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12aff6-3312-42b7-9f94-e62d2c40cc01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.466038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12aff6-3312-42b7-9f94-e62d2c40cc01 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.495041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 234/467] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:49:27 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:49:27.496300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c6cf-7a75-4de9-9364-2a2fb69dd282 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.496625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c6cf-7a75-4de9-9364-2a2fb69dd282 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.496863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c6cf-7a75-4de9-9364-2a2fb69dd282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.528187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2a12c6cf-7a75-4de9-9364-2a2fb69dd282 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:27.529354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2a12c6cf-7a75-4de9-9364-2a2fb69dd282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:27.594053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3643aab-8b6b-4838-9b57-51a1fa0bace1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.617594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 235/468] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:27.619363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c6cf-7a75-4de9-9364-2a2fb69dd282 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.642396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ee219-cc8d-48e9-9027-6e0ef846e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.643031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ee219-cc8d-48e9-9027-6e0ef846e22b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.643408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ee219-cc8d-48e9-9027-6e0ef846e22b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.647520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ee219-cc8d-48e9-9027-6e0ef846e22b None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.647520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ee219-cc8d-48e9-9027-6e0ef846e22b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:27.647520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ee219-cc8d-48e9-9027-6e0ef846e22b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.647520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ee219-cc8d-48e9-9027-6e0ef846e22b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.662257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 234/469] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:27.662896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:27.668841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62d82-7931-4688-aeb3-6ad19c2a3ec5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.669128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62d82-7931-4688-aeb3-6ad19c2a3ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.669388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62d82-7931-4688-aeb3-6ad19c2a3ec5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.675760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 235/470] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:27.676681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db62d82-7931-4688-aeb3-6ad19c2a3ec5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.721354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:27.780148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c6cf-7a75-4de9-9364-2a2fb69dd282 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.799529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db62d82-7931-4688-aeb3-6ad19c2a3ec5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:27.824051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d1ac6-6107-4ded-913d-f19ddc1a6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.825051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1ac6-6107-4ded-913d-f19ddc1a6e08 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.825505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1ac6-6107-4ded-913d-f19ddc1a6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.826218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1ac6-6107-4ded-913d-f19ddc1a6e08 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.827276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d1ac6-6107-4ded-913d-f19ddc1a6e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:27.827746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d1ac6-6107-4ded-913d-f19ddc1a6e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.829963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d1ac6-6107-4ded-913d-f19ddc1a6e08 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.837564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:27.838186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.838468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.838739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.839394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:27.840877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:27.854794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 236/471] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:27.860079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7fd31-a6f6-47e1-81e6-80d700aa0214 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:27.860573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7fd31-a6f6-47e1-81e6-80d700aa0214 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:27.861026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7fd31-a6f6-47e1-81e6-80d700aa0214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:27.872452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 236/472] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:49:27 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:49:27.874089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.889945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8ce7fd31-a6f6-47e1-81e6-80d700aa0214 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:27.892015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8ce7fd31-a6f6-47e1-81e6-80d700aa0214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:27.938314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:27.993298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 237/473] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:27.994012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7fd31-a6f6-47e1-81e6-80d700aa0214 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:27.997248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ad94-cc86-4e41-a34b-c7c6b91f4c3d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:28.066619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:28.067165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:28.067443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:28.067740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:28.068407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:28.072369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:28.105503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 237/474] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:49:27 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:49:28.111323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:28.171027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7fd31-a6f6-47e1-81e6-80d700aa0214 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:28.223480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:28.224546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:28.225009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:28.225493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:28.238323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:28.240198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:28.260150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:28.261104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:28.261942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:28.261942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:28.261942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:28.263705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 238/475] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:28.270484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:28.275414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:28.396878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f2ce1-066d-4f3b-9afe-806af9aa509d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:28.404236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74c2e4-440f-4f1b-9a11-f939ee62c7a9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:28.405621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:28.418297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:28.447721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:28.448640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:28.449017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:28.449360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:28.450453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:28.452229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:28.455039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:28.456368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:28.456849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:28.457303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:28.458225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:28.458712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:28.460298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:28.463431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.sql.core [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 22:49:28.463431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 22:49:28.463431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [parameters: {'id': '06dc8e44e0ff4902aef568c3990d2827', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 22:49:28.463431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81404) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 22:49:28.475349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 238/476] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application conn.query(q) Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result.read() Sep 09 22:49:28.481153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 22:49:28.482035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application next(self.gen) Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application yield resource Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application next(self.gen) Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise self.value Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application session.commit() Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 22:49:28.482940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.session.flush() Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application compat.raise_( Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 22:49:28.484445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 22:49:28.485642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application conn.query(q) Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application result.read() Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 22:49:28.487373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [parameters: {'id': '06dc8e44e0ff4902aef568c3990d2827', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:28.489464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 22:49:28.494801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:28.496869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 239/477] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 22:49:28 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:28.497006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:28.497151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:28.610733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b6447-7e1b-4f9e-a050-bd89da9d348d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:28.624571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440ccfa-4b36-43a0-9c84-b24860f289c8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:28.670629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:28.671194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:28.671474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:28.671771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:28.676226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c69143-a257-42ec-a1d8-86ea13c5b2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:28.676857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c69143-a257-42ec-a1d8-86ea13c5b2cb None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:28.677134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c69143-a257-42ec-a1d8-86ea13c5b2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:28.677402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c69143-a257-42ec-a1d8-86ea13c5b2cb None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:28.678078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c69143-a257-42ec-a1d8-86ea13c5b2cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:28.679614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c69143-a257-42ec-a1d8-86ea13c5b2cb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:28.691420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:28.703220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:28.704383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:28.705377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 240/478] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:49:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:28.707288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:28.731314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 239/479] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:49:28 2022] POST /identity/v3/users => generated 249 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:49:28.754613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:28.875588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd9bd-5873-4f91-a8b6-494c9252c4ad None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:28.926318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e6aae-51c8-44bb-9204-186bce789d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:28.927187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e6aae-51c8-44bb-9204-186bce789d39 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:28.927635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e6aae-51c8-44bb-9204-186bce789d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:28.928053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e6aae-51c8-44bb-9204-186bce789d39 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:28.928861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e6aae-51c8-44bb-9204-186bce789d39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:28.929304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e6aae-51c8-44bb-9204-186bce789d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:28.931405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e6aae-51c8-44bb-9204-186bce789d39 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:28.942829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 241/480] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:30.325714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694ffe8-447d-401b-8fbe-bbf3356ce167 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:30.326252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694ffe8-447d-401b-8fbe-bbf3356ce167 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:30.326551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694ffe8-447d-401b-8fbe-bbf3356ce167 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:30.328289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 240/481] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:30.333665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e37fd-df17-49e1-ae9b-2df1f40ecc71 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:30.334065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e37fd-df17-49e1-ae9b-2df1f40ecc71 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:30.334237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e37fd-df17-49e1-ae9b-2df1f40ecc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:30.353414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-204e37fd-df17-49e1-ae9b-2df1f40ecc71 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:30.354621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-204e37fd-df17-49e1-ae9b-2df1f40ecc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:30.428220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 242/482] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:30.723885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694ffe8-447d-401b-8fbe-bbf3356ce167 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:30.766168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:30.814452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d408e-b064-4fd0-82b5-758d3e581416 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:30.814835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d408e-b064-4fd0-82b5-758d3e581416 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:30.815109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d408e-b064-4fd0-82b5-758d3e581416 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:30.816716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 243/483] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:30.822437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ff92-5c3e-41dc-abc3-ec278957fd6b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:30.822827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ff92-5c3e-41dc-abc3-ec278957fd6b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:30.823106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ff92-5c3e-41dc-abc3-ec278957fd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:30.842684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3f68ff92-5c3e-41dc-abc3-ec278957fd6b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:30.844053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3f68ff92-5c3e-41dc-abc3-ec278957fd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:30.862533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:30.885983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694ffe8-447d-401b-8fbe-bbf3356ce167 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:30.924455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:30.925080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:30.925354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:30.925620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:30.936026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 244/484] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:30.936573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:30.937389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44697c2b-bbd2-4b4a-9dde-bc8ffa6ea0a2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:30.937670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44697c2b-bbd2-4b4a-9dde-bc8ffa6ea0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:30.937936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44697c2b-bbd2-4b4a-9dde-bc8ffa6ea0a2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:30.939310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 245/485] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:30.941856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:30.942743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:30.943597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 241/486] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:30.944974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeb7c7-474f-4933-baa0-08d410d73f58 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:30.944974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeb7c7-474f-4933-baa0-08d410d73f58 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:30.945250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeb7c7-474f-4933-baa0-08d410d73f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:30.948686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:30.963738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-87aeb7c7-474f-4933-baa0-08d410d73f58 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:30.965104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-87aeb7c7-474f-4933-baa0-08d410d73f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:31.035460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:31.043672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 246/487] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:31.050932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6a328-1c9a-4a35-8ba5-0c79de362b5e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.085312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.086074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.086457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.087352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.088137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:31.088535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:31.090021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:31.107371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 242/488] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:31.114295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeb7c7-474f-4933-baa0-08d410d73f58 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.225776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:31.248116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeb7c7-474f-4933-baa0-08d410d73f58 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.285609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.311594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.312710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.313460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.313963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.326219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:31.327993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:31.345664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:31.347008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:31.347982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:31.347982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:31.347982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:31.350333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 247/489] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:31.357014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.472730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e042a-03e6-4117-8ad5-a5198b33411e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.488567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271871e4-196b-400b-b176-cb45fb994031 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.518867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.520037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.520539 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.521009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.532103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.532733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.533017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.533291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.533949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:31.534251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:31.535776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:31.536476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:31.537451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:31.546136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:31.547136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:31.547956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:31.554638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 248/490] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:31.554638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.555361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 243/491] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:31.556453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.573235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:31.679216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7fb4d-947c-4bd9-a42d-a5e26c0e7f91 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.681650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df54f0-50d1-4494-95f9-cdb0bf0d5ce9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.720179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.720804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.721082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.721358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.729023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91bc6d-6a8e-416d-8dd6-31abcaa0a37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.729023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91bc6d-6a8e-416d-8dd6-31abcaa0a37d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.729023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91bc6d-6a8e-416d-8dd6-31abcaa0a37d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.729023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91bc6d-6a8e-416d-8dd6-31abcaa0a37d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.736417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91bc6d-6a8e-416d-8dd6-31abcaa0a37d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:31.736694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:31.737896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91bc6d-6a8e-416d-8dd6-31abcaa0a37d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:31.741758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:31.742631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 249/492] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:31.748538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ea91bc6d-6a8e-416d-8dd6-31abcaa0a37d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:31.757148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:31.758052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:31.759479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 244/493] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:31.760790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9c51-0f75-4ae4-9451-7bf16e80363f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.760790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9c51-0f75-4ae4-9451-7bf16e80363f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.760986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b9c51-0f75-4ae4-9451-7bf16e80363f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.766192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 245/494] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:31.766192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b9c51-0f75-4ae4-9451-7bf16e80363f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.810229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:31.831154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:31.856676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ce7ad-b97e-4139-8ff9-50edde70862c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.866760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b9c51-0f75-4ae4-9451-7bf16e80363f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:31.889006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ddd4a-c134-4240-a405-2390097feb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.889604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ddd4a-c134-4240-a405-2390097feb24 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.889881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ddd4a-c134-4240-a405-2390097feb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.890149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ddd4a-c134-4240-a405-2390097feb24 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.890810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ddd4a-c134-4240-a405-2390097feb24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:31.891108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ddd4a-c134-4240-a405-2390097feb24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:31.892521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ddd4a-c134-4240-a405-2390097feb24 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:31.899971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302729da-2395-474a-9633-2bf6c74466e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:31.900568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302729da-2395-474a-9633-2bf6c74466e1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.900848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302729da-2395-474a-9633-2bf6c74466e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.901120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302729da-2395-474a-9633-2bf6c74466e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.901417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 250/495] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:31.901786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302729da-2395-474a-9633-2bf6c74466e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:31.902089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302729da-2395-474a-9633-2bf6c74466e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:31.902575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e237d27-baf4-4a08-9b5c-1409214191b2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.902862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e237d27-baf4-4a08-9b5c-1409214191b2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.903136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e237d27-baf4-4a08-9b5c-1409214191b2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.903462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302729da-2395-474a-9633-2bf6c74466e1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:31.907542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 251/496] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:31.907542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e237d27-baf4-4a08-9b5c-1409214191b2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:31.912944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 246/497] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:31.914135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef8d8c-944d-42d8-ae26-6a9217852425 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:31.914425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef8d8c-944d-42d8-ae26-6a9217852425 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:31.914764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef8d8c-944d-42d8-ae26-6a9217852425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:31.932645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a6ef8d8c-944d-42d8-ae26-6a9217852425 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:31.933853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a6ef8d8c-944d-42d8-ae26-6a9217852425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:31.998877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 247/498] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:31.999634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef8d8c-944d-42d8-ae26-6a9217852425 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.022924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e237d27-baf4-4a08-9b5c-1409214191b2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.040292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.058957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.059984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a39e9-5aa0-4833-ac1c-b9399d50d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.060671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a39e9-5aa0-4833-ac1c-b9399d50d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.060926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a39e9-5aa0-4833-ac1c-b9399d50d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.061228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a39e9-5aa0-4833-ac1c-b9399d50d7e9 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.061987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a39e9-5aa0-4833-ac1c-b9399d50d7e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:32.062292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a39e9-5aa0-4833-ac1c-b9399d50d7e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.063741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a39e9-5aa0-4833-ac1c-b9399d50d7e9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.101106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 252/499] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:32.102364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ceb93-e24b-4966-b317-dc27528b16ce None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.102647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ceb93-e24b-4966-b317-dc27528b16ce None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.102917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ceb93-e24b-4966-b317-dc27528b16ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.103168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef8d8c-944d-42d8-ae26-6a9217852425 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.120912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f18ceb93-e24b-4966-b317-dc27528b16ce None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:32.122101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f18ceb93-e24b-4966-b317-dc27528b16ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:32.141013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.141888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.142168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.142438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.153586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.155041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:32.169466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:32.170824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:32.171742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:32.171742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:32.171742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:32.171742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 248/500] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:32.171977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.199640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 253/501] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:32.200484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ceb93-e24b-4966-b317-dc27528b16ce None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.292218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a725da5-df77-4585-97b9-194ba1b81bc7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.305656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.311426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ceb93-e24b-4966-b317-dc27528b16ce None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.314517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.328466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.329097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.329374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.329644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.340375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.341827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.349182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.350182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.350466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.350893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:32.356260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:32.357154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:32.358012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:32.358012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:32.358012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:32.361965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 249/502] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:32.362792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.423193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.424690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.466981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743339-d5d1-44d7-9abb-d80e9eae982c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.500950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.501561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.501817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.502086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.502724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:32.503024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.504426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.515798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 250/503] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:32.516573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.533092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 254/504] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:32.533092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.534151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.541345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.613862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e02db-3dc0-4a5f-96c1-350c08832106 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.627873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2bf83-b3bf-4f65-a0e7-540b4b6254c8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.647012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.647669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.647951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.648222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.658613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.660140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.663432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.664050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.664327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.664593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.665780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:32.666075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.667408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:32.675285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:32.676367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:32.677205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:32.677205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:32.677205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:32.679169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 255/505] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:32.679629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 251/506] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:32.679991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.680954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.745063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.760397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.783699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee17f9-6105-4ba3-aa16-9a30e823bc78 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.784039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401df139-4e84-4ede-b5f4-82952c028059 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.816825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.817105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.817442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.817653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.817773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.817936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.818026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.818209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.825497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.826954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.828499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:32.833772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:32.834790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 256/507] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:32.835648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:32.838758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:32.839670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:32.840579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 252/508] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:32.841138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.937759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289379e9-c991-4ca8-b911-decc20eb5184 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.938156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1011-8686-4bc2-8fab-5b12a18f16aa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:32.960210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.969421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.970010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.970285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.970551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.971177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:32.971323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:32.971473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.971977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:32.972259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:32.972534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:32.972857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.975816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:32.983783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:32.985259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:32.986044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 253/509] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:32.986991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:32.999581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:33.000674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:33.001705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:33.001705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:33.001705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:33.001705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 257/510] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:33.001705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.086062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79459c7f-e543-41b0-83ae-c5e511382127 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.100890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d25ba-d5be-40f7-9428-9ea525c60cca None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.124759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.125610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.125881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.126167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.133225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.135725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.137338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.137937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.138218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.138487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.139135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:33.139434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.140806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.148480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 258/511] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:33.149256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:33.149463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:33.150335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:33.151231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 254/512] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:33.151833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.192294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.208088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.251798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7c1e2-2b24-4c13-a6ba-d2988059daf2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.252151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e149-3b5d-47a6-a84b-420448fd64d9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.284724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.285302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.285459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.285606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.285883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.285979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.286164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.286435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.286521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.287147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:33.287454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.287986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.288888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.306309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 259/513] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:33.307216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.319734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 255/514] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:49:33 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:49:33.319734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.425153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043111c0-e972-4e9a-8912-ddabb6a75ea1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.431315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.440705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d877e-4d46-4e00-b5b6-d0357717cd8e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.444486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.469123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.469771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.470050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.470316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.482754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.484332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.488087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.488583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.488860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.489131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.489762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:33.490060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.491412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:33.501953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:33.503185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:33.504086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:33.504086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:33.504086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:33.504086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 256/515] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:33.504086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.509776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 260/516] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:33.510952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.622976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40a722-3c54-4493-ae67-f323d656c570 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.625012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75967d91-f1b0-4c25-9ad7-de1c9d9af245 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.660110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.660713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.660990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.661258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.661598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.662472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.662749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.663018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.663764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:33.664068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.666090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.689401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.703838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 257/517] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:33.704774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.733879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.750521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.752019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.812013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1033-e1f7-4d28-87e9-af3be39b0cc8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.849934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:33.850546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:33.850822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:33.851091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:33.851758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:33.852065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:33.853466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:33.859326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 261/518] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:33.860378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.870228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 258/519] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:33.871110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:33.924976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.961065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:33.972030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02a349-b238-4684-8e86-482c1c6afb33 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:33.981340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7a0c1-6c2d-45d3-8065-2d9fae05c640 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.008904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.009638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.010063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.010335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.014775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.015385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.015694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.015967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.017551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.019276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:34.032986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:34.033887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:34.034740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 262/520] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:34.035856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.093535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.095075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.139507 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957aef3-f8d0-4826-967b-9f035f4324c9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.155456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:34.174016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.174637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.174913 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.175186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.180191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:34.186320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.187795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.201114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 259/521] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:34.201886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:34.210720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:34.211934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:34.212930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:34.212930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:34.212930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:34.212930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 263/522] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:34.213322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.314062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9fff6-c548-44c6-8be3-00fbc51dacca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.323566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1be88-2470-4747-9a0c-e9f5fc053d4c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.347457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.348129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.348395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.348663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.356895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.358345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.363640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.364344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.364878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.365357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.366132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:34.366537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.368113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:34.370156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:34.371176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 260/523] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:34.372292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.398199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 264/524] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:34.398199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.422429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:34.426634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:34.484848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e6b7-f38e-497e-962b-be3d7e738c89 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.502154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58cd46-4a84-49dd-aaaf-f9e9a9a0d45f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.519727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.520362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.520637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.520908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.537968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.538570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.538849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.539120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.540318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:34.540617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.541925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.542031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.543388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.558265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 265/525] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:34.558265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:34.564140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:34.565080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:34.565938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:34.565938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:34.565938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:34.565938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 261/526] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:34.566175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.660578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:34.666654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:34.674537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991a856-4356-43a9-932d-2d30535d024e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.675311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eff506-df9f-4c52-bc8d-cc68e3eac695 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.710034 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.710641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.710920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.711193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.711995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=786e4e7b1c5147e093693767b229e52b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:34.712301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.713737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.718879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.719294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 262/527] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/role_assignments?user.id=786e4e7b1c5147e093693767b229e52b&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:34.719968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.720142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.720712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.721077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.721900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:34.722319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.723913 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.756158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 266/528] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:34.756158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.869055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944277c0-8efe-4f3c-969a-b32f57eed05a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.873993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b57c1-eb38-4857-b9eb-450a2007d331 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:34.913865 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.914479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.916664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.917737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.917737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:34.919605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.919727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:34.919727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:34.920165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.920389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:34.920750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:34.933887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:34.934722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 267/529] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:34.935650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:34.941312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:34.946464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:34.963513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:34.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 263/530] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:34.965678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:35.010626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:35.057822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42a3a-7dc0-432d-849c-162d2daa33a5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.110986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:35.111623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.111907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.112174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.147121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4429fc8-3084-4e75-8314-602e104554d8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.220119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:35.220811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.221101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.221366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.221613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:35.223087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:35.229093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:35.230542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:35.234087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:35.242580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:35.244146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:35.246320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:35.247824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 264/531] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:35.248693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:35.344300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:35.368787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4849483-92c8-4ab9-baa9-f3ec9b161606 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.405262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 268/532] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:35.406143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:35.425929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:35.426584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.426871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.427139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.429761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:35.430067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:35.432355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:35.471518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 265/533] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:35.473696 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:35.616376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3a2c6-9733-4a77-9b53-6e399ce159ff None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.621237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bea1a7-407a-408d-92a0-8429c4703c7b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.628661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:35.657387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c5d49-e7aa-46b7-a333-f6b418c54b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:35.658020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c5d49-e7aa-46b7-a333-f6b418c54b33 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.658294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c5d49-e7aa-46b7-a333-f6b418c54b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.658561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c5d49-e7aa-46b7-a333-f6b418c54b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.659249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:35.659854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.660140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.660433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.661175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:35.661486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:35.662912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:35.666451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:35.671144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c5d49-e7aa-46b7-a333-f6b418c54b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:35.673021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c5d49-e7aa-46b7-a333-f6b418c54b33 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:35.681535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 269/534] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:35.681535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b34c5d49-e7aa-46b7-a333-f6b418c54b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:35.689739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:35.690640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:35.691466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:35.691466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:35.691466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:35.691466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 266/535] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:35.692260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd789-2035-46aa-8e52-40952688e60f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.692260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd789-2035-46aa-8e52-40952688e60f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.692512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fd789-2035-46aa-8e52-40952688e60f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.697704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 267/536] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:35.697704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fd789-2035-46aa-8e52-40952688e60f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:35.801292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1075f-cb42-4d5d-8700-35b2964451a2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.811317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fd789-2035-46aa-8e52-40952688e60f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.839767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:35.840488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.840771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.841042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.841760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=56228e09c8b742b29e49c40d1a39870f) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:35.842100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:35.843506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:35.844898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e604c1-e751-438d-8305-fb7441c378be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:35.845491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e604c1-e751-438d-8305-fb7441c378be None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.845769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e604c1-e751-438d-8305-fb7441c378be None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.846035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e604c1-e751-438d-8305-fb7441c378be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.846740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e604c1-e751-438d-8305-fb7441c378be None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=6cf808a1f5dd45bea0a3a81131532b2b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:35.847041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e604c1-e751-438d-8305-fb7441c378be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:35.848488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e604c1-e751-438d-8305-fb7441c378be None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:35.859710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 270/537] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:35 2022] GET /identity/v3/role_assignments?user.id=56228e09c8b742b29e49c40d1a39870f&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:35.859710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:35.865046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 268/538] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:35 2022] GET /identity/v3/role_assignments?user.id=6cf808a1f5dd45bea0a3a81131532b2b&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:35.866438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2cc7-22d4-4b0d-9f8e-0fe1c317ecf4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:35.866667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2cc7-22d4-4b0d-9f8e-0fe1c317ecf4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:35.866937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2cc7-22d4-4b0d-9f8e-0fe1c317ecf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:35.889533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0bcd2cc7-22d4-4b0d-9f8e-0fe1c317ecf4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:35.891085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0bcd2cc7-22d4-4b0d-9f8e-0fe1c317ecf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:35.895275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:35.969759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:35.979567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2784e-8425-43cc-b2b7-b85f5db8aa9d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:35.999202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 269/539] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:36.026471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:36.027097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:36.027378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:36.027730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:36.028386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:36.028789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:36.030280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:36.045580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 271/540] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:36.051431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2cc7-22d4-4b0d-9f8e-0fe1c317ecf4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:36.188015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2cc7-22d4-4b0d-9f8e-0fe1c317ecf4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:36.228076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:36.228717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:36.228996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:36.229265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:36.236216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:36.237687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:36.250463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:36.251321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:36.252193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 270/541] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:36.257591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:36.352574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:36.362884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38e346-954c-4d8a-b785-da86f91b4da3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:36.398533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:36.399807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:36.399995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:36.400274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:36.400968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:36.401274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:36.402703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:36.405839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:36.433878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 272/542] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:36.440289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:36.503834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:36.611430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976c6f-3c9d-478e-8482-142642d72add None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:36.635370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bcf8c-e4d4-469b-b144-0f4047965207 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:36.649816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:36.651060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:36.651543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:36.652016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:36.662629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:36.669119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:36.670260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:36.671048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:36.674720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 271/543] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:36.680618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:36.719622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:36.721326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:36.721790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:36.722244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:36.723167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=c0e10b1ef7454167bd1c606ef4f70bf3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:36.723691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:36.725475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:36.744283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 273/544] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:36 2022] GET /identity/v3/role_assignments?user.id=c0e10b1ef7454167bd1c606ef4f70bf3&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:36.836138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673297-7b31-48d8-8c88-5c70c0a5b595 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:36.844839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:36.896474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ca5dd-0c1a-440c-9329-8e7c52169f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:36.897101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ca5dd-0c1a-440c-9329-8e7c52169f86 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:36.897377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ca5dd-0c1a-440c-9329-8e7c52169f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:36.897644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ca5dd-0c1a-440c-9329-8e7c52169f86 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:36.898324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ca5dd-0c1a-440c-9329-8e7c52169f86 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:36.898623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ca5dd-0c1a-440c-9329-8e7c52169f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:36.900649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ca5dd-0c1a-440c-9329-8e7c52169f86 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:36.916497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 272/545] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:36.922621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:36.957839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:37.037789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76afe75-fb66-40c7-a28e-c175ced1aa0c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:37.038100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76afe75-fb66-40c7-a28e-c175ced1aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:37.038345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76afe75-fb66-40c7-a28e-c175ced1aa0c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:37.040948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 273/546] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:37.045195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32abea-5b3f-4df7-a405-fa73226d6f20 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:37.045479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32abea-5b3f-4df7-a405-fa73226d6f20 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:37.045745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32abea-5b3f-4df7-a405-fa73226d6f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:37.064020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235225fc-6556-4c6a-ac7a-2defe2563911 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:37.079774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9b32abea-5b3f-4df7-a405-fa73226d6f20 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:37.082178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b32abea-5b3f-4df7-a405-fa73226d6f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:37.130558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:37.131257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:37.131559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:37.131831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:37.164345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:37.167345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:37.193434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:37.194382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:37.195209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:37.195209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:37.195209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:37.199356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 274/547] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:37.206362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:37.212180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 274/548] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:37.304948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:37.351689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7f12e-377a-4db4-8057-ae5d961cb76f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:37.386909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:37.387536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:37.387821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:37.388137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:37.388778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:37.389134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:37.390524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:37.403682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 275/549] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:37.409249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32abea-5b3f-4df7-a405-fa73226d6f20 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:37.525016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32abea-5b3f-4df7-a405-fa73226d6f20 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:37.527841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:37.538969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:37.586557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:37.587080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:37.587359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:37.587658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:37.600827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:37.602796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:37.624286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:37.625804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:37.626801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:37.634001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 275/550] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:37.636318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:37.738448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660aa0-7861-4f89-8d09-f37a808fd7d7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:37.788983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:37.804463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:37.805617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:37.805908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:37.806175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:37.825688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ef1cb-12e8-4ce4-bc4d-429b0b9eb3fc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:37.827854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:37.832712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:37.833621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:37.834613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:37.835232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 276/551] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:37.841715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:37.890924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:37.891556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:37.891839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:37.892111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:37.892843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:37.893148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:37.894598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:37.928431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 276/552] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 22:49:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:37.934533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:37.959889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:38.035749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7596f-38ef-41f2-ac4a-cf7259aec691 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.061742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84860f-ef41-47e3-89d6-3466f22eac17 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.071471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.072167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.072453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.072780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.128881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.129426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.129712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.129989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.130661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:38.130965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.132413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.146096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.146960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 277/553] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:38.147703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.156502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:38.179865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:38.252843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 277/554] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:38.258889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:38.273072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:38.363113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38001b30-6518-443e-9686-0bf09dda6204 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.379981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:38.395429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1eb46-635a-4d31-a014-c2c9229ba633 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.412111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.412753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.413033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.413364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.434221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.435712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:38.462817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:38.463834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:38.464689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:38.464689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:38.464689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:38.466121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ad629-7e3a-48eb-a56e-07f802e2323a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.466752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad629-7e3a-48eb-a56e-07f802e2323a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.467030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad629-7e3a-48eb-a56e-07f802e2323a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.467296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad629-7e3a-48eb-a56e-07f802e2323a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.469840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 278/555] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:38.479056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:38.488563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ad629-7e3a-48eb-a56e-07f802e2323a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.490286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ad629-7e3a-48eb-a56e-07f802e2323a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ae0ad629-7e3a-48eb-a56e-07f802e2323a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:38.506131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:38.507757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:38.509209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:38.509209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:38.509209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:38.515436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 278/556] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:38.516922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c68a1b-3bf5-443d-a7fb-559118591b16 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.517414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c68a1b-3bf5-443d-a7fb-559118591b16 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.517878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c68a1b-3bf5-443d-a7fb-559118591b16 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.519712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 279/557] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:38.520595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c68a1b-3bf5-443d-a7fb-559118591b16 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:38.587328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831084f6-f73a-4c1f-a8f6-0710529ef5eb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.622536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321b06c-0af0-4e43-a351-ddd8995463df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.623232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b06c-0af0-4e43-a351-ddd8995463df None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.623579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b06c-0af0-4e43-a351-ddd8995463df None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.623811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b06c-0af0-4e43-a351-ddd8995463df None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.624545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321b06c-0af0-4e43-a351-ddd8995463df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:38.624859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321b06c-0af0-4e43-a351-ddd8995463df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.626320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321b06c-0af0-4e43-a351-ddd8995463df None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.635982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:38.640167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c68a1b-3bf5-443d-a7fb-559118591b16 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.645002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 279/558] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:38.646526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519975-c8b3-46cf-8561-22018dca34c4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.646820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519975-c8b3-46cf-8561-22018dca34c4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.647098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67519975-c8b3-46cf-8561-22018dca34c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.667998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-67519975-c8b3-46cf-8561-22018dca34c4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:38.669214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67519975-c8b3-46cf-8561-22018dca34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:38.686335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.687254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.689159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.689676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.690397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:38.690823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.692778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.702471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:38.713559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 280/559] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:38.714649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:38.764177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 280/560] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:38.765031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67519975-c8b3-46cf-8561-22018dca34c4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:38.833853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e1ea9-47fd-4b15-b87b-555f98f917f8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.875160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cfcaa-a28a-4ba3-81a9-b09f324cfb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.876175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cfcaa-a28a-4ba3-81a9-b09f324cfb54 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.876814 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cfcaa-a28a-4ba3-81a9-b09f324cfb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.877294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cfcaa-a28a-4ba3-81a9-b09f324cfb54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.884876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cfcaa-a28a-4ba3-81a9-b09f324cfb54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.886648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cfcaa-a28a-4ba3-81a9-b09f324cfb54 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.894620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67519975-c8b3-46cf-8561-22018dca34c4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3e6cfcaa-a28a-4ba3-81a9-b09f324cfb54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:38.900533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:38.901423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:38.902334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:38.905125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 281/561] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:38.906621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1965d0a-0f7d-4569-bf77-b976a34240ae None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.907114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1965d0a-0f7d-4569-bf77-b976a34240ae None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.907622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1965d0a-0f7d-4569-bf77-b976a34240ae None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.909422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 282/562] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:38.910415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1965d0a-0f7d-4569-bf77-b976a34240ae None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:38.935199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:38.943689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650c1b6-0a6d-4585-8cc3-33df4e593b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:38.944339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c1b6-0a6d-4585-8cc3-33df4e593b3a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.944626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c1b6-0a6d-4585-8cc3-33df4e593b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.945151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c1b6-0a6d-4585-8cc3-33df4e593b3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:38.955784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650c1b6-0a6d-4585-8cc3-33df4e593b3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:38.957299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650c1b6-0a6d-4585-8cc3-33df4e593b3a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9650c1b6-0a6d-4585-8cc3-33df4e593b3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:38.973290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:38.974276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:38.975210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 281/563] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:49:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:38.976348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d53a-4858-4099-a675-286d00dbec2e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:38.976449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d53a-4858-4099-a675-286d00dbec2e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:38.976689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d53a-4858-4099-a675-286d00dbec2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.000387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e23d53a-4858-4099-a675-286d00dbec2e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:39.002105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e23d53a-4858-4099-a675-286d00dbec2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:39.016834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:39.033166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1965d0a-0f7d-4569-bf77-b976a34240ae None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.070073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.071187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.071477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.071777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.072457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:39.072757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.074180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.083989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 282/564] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:39.083989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23d53a-4858-4099-a675-286d00dbec2e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.089846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 283/565] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:39.090655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.200631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:39.208899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23d53a-4858-4099-a675-286d00dbec2e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.231477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd114de7-8e57-47dd-accf-196d815a0bed None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.248526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.249342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.249742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.250020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.251118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:39.251240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.252637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.272132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 283/566] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:39.272982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.276176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.276808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.277319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.277425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.279627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:39.298210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:39.305720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:39.306706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:39.307569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:39.309158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 284/567] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:39.309863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.393726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477b41-307a-4486-9012-0eb88b76e1c3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.411620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d0104-23f3-4b58-952a-7280e655516e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.428304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.428910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.429240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.429511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.430759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=e7ceb38886d34f9d8e59b2871989446c) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:39.431067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.432538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.437755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 284/568] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/role_assignments?user.id=e7ceb38886d34f9d8e59b2871989446c&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:39.438527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.449484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd463a1-d715-4bc4-948f-654618329a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.450476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd463a1-d715-4bc4-948f-654618329a30 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.450809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd463a1-d715-4bc4-948f-654618329a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.451014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd463a1-d715-4bc4-948f-654618329a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.461167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:39.464346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd463a1-d715-4bc4-948f-654618329a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.466689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd463a1-d715-4bc4-948f-654618329a30 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-8cd463a1-d715-4bc4-948f-654618329a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:39.482519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:39.483777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:39.484874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:39.484874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:39.484874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:39.487713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 285/569] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:39.489470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b1db2-2417-4a0b-a2f9-02700450a1e0 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.490156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b1db2-2417-4a0b-a2f9-02700450a1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.490581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b1db2-2417-4a0b-a2f9-02700450a1e0 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.495620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 286/570] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:39.495620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b1db2-2417-4a0b-a2f9-02700450a1e0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.532793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:39.570054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18305e-3e66-4fa8-8723-c4accc4b0451 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.607737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.608355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.608632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.608897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.609561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:39.610318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.611215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.616337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b1db2-2417-4a0b-a2f9-02700450a1e0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.622174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 285/571] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:39.622174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.654164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7157c-7449-486e-b34a-90a2391b4d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.654785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7157c-7449-486e-b34a-90a2391b4d14 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.655063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7157c-7449-486e-b34a-90a2391b4d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.655330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7157c-7449-486e-b34a-90a2391b4d14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.667870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7157c-7449-486e-b34a-90a2391b4d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0fc7157c-7449-486e-b34a-90a2391b4d14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:39.673378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:39.674474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:39.675589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 287/572] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:39.676280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674879a-4f8d-4e2d-bf8f-09adead7b037 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.676280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674879a-4f8d-4e2d-bf8f-09adead7b037 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.676482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674879a-4f8d-4e2d-bf8f-09adead7b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.699227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4674879a-4f8d-4e2d-bf8f-09adead7b037 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:39.700470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4674879a-4f8d-4e2d-bf8f-09adead7b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:39.744355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecba1d6-5b4b-4b39-aa3e-e63df3f5d560 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.763192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:39.774738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:39.781654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.782258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.782538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.782809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.783448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:39.783773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.795785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.795958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 288/573] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:39.795958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674879a-4f8d-4e2d-bf8f-09adead7b037 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.802371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 286/574] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:39.803282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.919755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674879a-4f8d-4e2d-bf8f-09adead7b037 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.939432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aa5f4-72a3-4dbb-a1f8-ffaf64cbe604 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:39.957001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.957641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.957923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.958203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.969607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.971163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.981192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:39.982140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:39.982371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:39.982648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:39.983316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:39.983661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:39.985038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:39.986989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:39.987916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:39.988798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:39.988798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:39.988798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:39.992028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 289/575] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:39.993038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:39.993596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 287/576] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:39.998683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.043827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.065766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.112366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21bf85-24ea-4c39-a0a1-de1b1c236c24 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.123393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8854b-1559-4296-9e73-34a4b051dd6e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.146625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.147257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.147567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.147847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.155156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.156659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.158337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.159196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.159306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.159534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:40.168552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:40.169616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 290/577] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:40.170763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.176757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.179241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:40.199195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:40.200197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:40.201073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:40.201073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:40.201073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:40.201073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 288/578] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:40.201902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.285425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.293343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102fffc-3a8c-464b-b646-ea5fbb5cda3e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.307801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf25ce-9c05-47c8-871c-5e9310116e5d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.310441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.332699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.333312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.333593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.333862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.334506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:40.334808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.336209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.349273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.349897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.354531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.354679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 291/579] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:40.354789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.354877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.364455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:40.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:40.371968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:40.372807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 289/580] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:40.373574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.465779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190217a2-2a42-453a-9292-e10ab8fd56ea None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.474273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed6b8d-d566-4546-b2cd-29134fee8541 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.500742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570684e-5618-40b4-823e-166936935df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.501642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570684e-5618-40b4-823e-166936935df9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.501928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570684e-5618-40b4-823e-166936935df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.502212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570684e-5618-40b4-823e-166936935df9 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.502851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570684e-5618-40b4-823e-166936935df9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:40.503148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570684e-5618-40b4-823e-166936935df9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.504595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570684e-5618-40b4-823e-166936935df9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.509505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.510107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.510390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.510663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.511302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:40.511709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.513070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.522454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 292/581] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:40.523385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570684e-5618-40b4-823e-166936935df9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.530566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 290/582] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:40.530566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.551991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.559069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.641107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66be2b2-2800-4844-9ffd-d4bc2c3c0984 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.646932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570684e-5618-40b4-823e-166936935df9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.677286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.677897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.678175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.678445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.679098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:40.679399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.680819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.684109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.684743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.685026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.685301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.688760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 291/583] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:40.689461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.699353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.701270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:40.720109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:40.721101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 293/584] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:40.722057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.784406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.801900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf95eb-f65d-454e-b733-fa9a666808bb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.808675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:40.835273 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b88ae-2df4-49f7-bd23-ea0c99e8f112 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:40.840487 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.841156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.841438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.841713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/786e4e7b1c5147e093693767b229e52b/roles/8d7867fc676d4963a98f9eaed5f9f9f6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.857173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=786e4e7b1c5147e093693767b229e52b, role_id=8d7867fc676d4963a98f9eaed5f9f9f6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.858636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.872835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:40.873467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:40.873741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:40.874044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:40.876677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 292/585] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:49:40 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/786e4e7b1c5147e093693767b229e52b/roles/8d7867fc676d4963a98f9eaed5f9f9f6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:40.879116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:40.883274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:40.884942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:40.899771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:40.901165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 22:49:40.903149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:40.905912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 294/586] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:40.906923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.043579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79e973-deb4-4fe1-b523-c21df9f874fd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.055593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b00438-bae6-4c4c-b47c-9f765febb8b9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.092911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.093940 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.094228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.094504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.096456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.097073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.097361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.097637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.098059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.098712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:41.099023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.100523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.102969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.108244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.110366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.125824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 295/587] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:41.125824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:41.142822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:41.143846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:41.144745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:41.144745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:41.144745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:41.145898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 293/588] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:41.145898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.240050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f24c-0fa9-4e2e-949e-d7f6e9d9b2b3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.277466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.278081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.278411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.278693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.279426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:41.279764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.281267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.284896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249e620-fc08-4d00-8e00-37d9e30119d2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.305137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 296/589] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:41.305137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.334200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.335895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.336465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.337335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.354185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.355637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.364960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:41.371152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:41.372177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:41.373080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:41.373080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:41.373080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:41.377109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 294/590] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:41.378067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.428109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.437347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ab661-c7cd-42d7-8eb6-a8f10fc7fa58 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.484385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.485564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.486296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.487168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.488325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:41.488847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.489780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b145990-5612-412f-92a8-ca1defb3106f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.490667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.509403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 297/591] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:41.511319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.533806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.534438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.534813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.535091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.546726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.548320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:41.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:41.564439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:41.565328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:41.565328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:41.565328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:41.569235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 295/592] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:41.570071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.647570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086da7b-527e-406b-8273-f00028c05fe5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.658393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.667767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.678582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6dcd-69aa-4cf6-98fb-612b09290c96 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.684594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.685210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.685491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.685765 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.686577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:41.686740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.688107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.706382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 298/593] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:41.706382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.719381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ebf1d-cf18-46bd-acc2-8aed3b425658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.720197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ebf1d-cf18-46bd-acc2-8aed3b425658 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.720479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ebf1d-cf18-46bd-acc2-8aed3b425658 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.720749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ebf1d-cf18-46bd-acc2-8aed3b425658 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.728679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ebf1d-cf18-46bd-acc2-8aed3b425658 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.730142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ebf1d-cf18-46bd-acc2-8aed3b425658 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5c4ebf1d-cf18-46bd-acc2-8aed3b425658 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:41.743573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:41.744531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:41.745470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 296/594] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:41.746049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c06151-0fd3-4ab6-bb0d-3c43e6086149 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.751175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c06151-0fd3-4ab6-bb0d-3c43e6086149 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.751175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c06151-0fd3-4ab6-bb0d-3c43e6086149 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.753963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 297/595] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:41.754815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c06151-0fd3-4ab6-bb0d-3c43e6086149 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.820739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5f95e-99ad-4cc0-a571-af267f09c31b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.860408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.861027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.861312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.861606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.862254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:41.862563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.863963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.866366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c06151-0fd3-4ab6-bb0d-3c43e6086149 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:41.877381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 299/596] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:41.878449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:41.904607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef62cb5-ab49-445f-bd30-65a74d284d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:41.905162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef62cb5-ab49-445f-bd30-65a74d284d5b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.905442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef62cb5-ab49-445f-bd30-65a74d284d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.905711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef62cb5-ab49-445f-bd30-65a74d284d5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.909808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.912403 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef62cb5-ab49-445f-bd30-65a74d284d5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:41.913858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef62cb5-ab49-445f-bd30-65a74d284d5b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-eef62cb5-ab49-445f-bd30-65a74d284d5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:41.925752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:41.926666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:41.928222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 298/597] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:41.928708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2bb38-05e2-40c5-9897-5c9294545b25 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:41.928708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2bb38-05e2-40c5-9897-5c9294545b25 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:41.929012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2bb38-05e2-40c5-9897-5c9294545b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:41.948265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ddb2bb38-05e2-40c5-9897-5c9294545b25 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:41.949467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddb2bb38-05e2-40c5-9897-5c9294545b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:41.956898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:41.988824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fec2d-2f80-47d2-8834-962a2d15040f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.016221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 299/598] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:42.017042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2bb38-05e2-40c5-9897-5c9294545b25 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.036884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.038224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.038709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.039195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.041160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:42.041677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.043872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.068505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 300/599] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 9 22:49:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:42.070124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.144310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2bb38-05e2-40c5-9897-5c9294545b25 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.186224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:42.189778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.191647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.192357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.193074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.201915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9218948-c264-42b7-b132-3c8a4e84fa71 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.203113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.205051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:42.218123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:42.219023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:42.219946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:42.223889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 300/600] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:42.224711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.228096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:42.246576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.247536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.248053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.248548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.249485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:42.249988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.251838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.269401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 301/601] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:42.270577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.331162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010138a-5bd6-4146-b3e3-4dcfd88f7d43 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.364867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.365496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.365796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.366046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.375413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51568e3-f835-46bf-a6c8-ba411ade8dcd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.377422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.379840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:42.395873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:42.396802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:42.397672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:42.397672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:42.397672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:42.397672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 301/602] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:42.397672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.412731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.413342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.413631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.413894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.414676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:42.414989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.416434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.429486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 302/603] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:42.430805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.444220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:42.463627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:42.508904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e2fae-e546-4736-9ddf-7f37bb271a0f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.541818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.542480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.542758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.542859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d7228-ef1f-4717-b913-cd6c8a4b16f0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.543033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/56228e09c8b742b29e49c40d1a39870f/roles/8d7867fc676d4963a98f9eaed5f9f9f6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.559977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=56228e09c8b742b29e49c40d1a39870f, role_id=8d7867fc676d4963a98f9eaed5f9f9f6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.561443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.575045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 302/604] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:49:42 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/56228e09c8b742b29e49c40d1a39870f/roles/8d7867fc676d4963a98f9eaed5f9f9f6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:42.575958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.590439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.591506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.592319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.593409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.608558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:42.615044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:42.616314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:42.617340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:42.620203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 303/605] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:42.621193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.694855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41758b3f-436e-4a6e-9d48-d5728a24cb91 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.698338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:42.715640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:42.751649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.753300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.753738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.754443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.755269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:42.756021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.758911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.763317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee4d6e-a496-4477-825a-d9ed42ad0a28 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.774607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 303/606] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:42.776200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.805978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.806729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.807010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.807284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/6cf808a1f5dd45bea0a3a81131532b2b/roles/8d7867fc676d4963a98f9eaed5f9f9f6` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.832215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=6cf808a1f5dd45bea0a3a81131532b2b, role_id=8d7867fc676d4963a98f9eaed5f9f9f6)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.833701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:42.853886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 304/607] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:49:42 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/6cf808a1f5dd45bea0a3a81131532b2b/roles/8d7867fc676d4963a98f9eaed5f9f9f6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:42.854767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:42.922427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1333fa-3cf9-43bc-9352-600029a31b24 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:42.981408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:42.981959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:42.982238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:42.982509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:42.983176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:42.983478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:42.984864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:43.006534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 304/608] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:43.012551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:43.015012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470dd71-da94-4dc8-a107-6fcb1be6c83e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:43.044242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:43.062514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:43.063221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.063536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.063837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.073420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:43.074915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:43.088820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:43.089827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:43.090754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:43.091205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:43.092327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 305/609] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:43.100764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:43.216242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd6091-6f38-4ae4-9c5c-02565e8424f1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:43.253805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1016-aa75-4f83-9bd0-adf319f5dd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:43.254515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649a55-ac40-43c5-ae93-470facdad45e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:43.254807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1016-aa75-4f83-9bd0-adf319f5dd1e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.255097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1016-aa75-4f83-9bd0-adf319f5dd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.255604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1016-aa75-4f83-9bd0-adf319f5dd1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.267974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1016-aa75-4f83-9bd0-adf319f5dd1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:43.269497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1016-aa75-4f83-9bd0-adf319f5dd1e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6b3c1016-aa75-4f83-9bd0-adf319f5dd1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:43.288664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:43.289764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:43.290674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:43.290674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:43.290674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:43.290852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 305/610] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:43.292407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c2a1-c508-42f1-9219-d38d4bbd1547 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.292407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c2a1-c508-42f1-9219-d38d4bbd1547 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.292748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c2a1-c508-42f1-9219-d38d4bbd1547 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.293973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 306/611] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:43.296155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa879a-ffab-405c-a8dd-7beba8925b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:43.296772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa879a-ffab-405c-a8dd-7beba8925b0e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.297048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa879a-ffab-405c-a8dd-7beba8925b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.297320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa879a-ffab-405c-a8dd-7beba8925b0e None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.298084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa879a-ffab-405c-a8dd-7beba8925b0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:43.298397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa879a-ffab-405c-a8dd-7beba8925b0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:43.298983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723c2a1-c508-42f1-9219-d38d4bbd1547 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:43.299846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa879a-ffab-405c-a8dd-7beba8925b0e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:43.313940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 306/612] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:43.315339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaa022-8556-4922-9a61-ad79096e8dc2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.315656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaa022-8556-4922-9a61-ad79096e8dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.315932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaa022-8556-4922-9a61-ad79096e8dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.337841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-85aaa022-8556-4922-9a61-ad79096e8dc2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:43.339057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-85aaa022-8556-4922-9a61-ad79096e8dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:43.376586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:43.397017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:43.428745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723c2a1-c508-42f1-9219-d38d4bbd1547 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:43.435617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 307/613] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:43.436554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaa022-8556-4922-9a61-ad79096e8dc2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:43.466862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:43.467809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.468577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.469011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.469860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:43.470317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:43.471926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:43.485535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 307/614] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:43.490489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:43.602283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaa022-8556-4922-9a61-ad79096e8dc2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:43.660542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:43.661620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.662112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.662621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/c0e10b1ef7454167bd1c606ef4f70bf3/roles/8d7867fc676d4963a98f9eaed5f9f9f6` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.678606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=c0e10b1ef7454167bd1c606ef4f70bf3, role_id=8d7867fc676d4963a98f9eaed5f9f9f6)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:43.680424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:43.694307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87b01a-8845-4fcb-8b89-832dde392442 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:43.696475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 308/615] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:49:43 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/c0e10b1ef7454167bd1c606ef4f70bf3/roles/8d7867fc676d4963a98f9eaed5f9f9f6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:43.764831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:43.774235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:43.774886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:43.775198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:43.775471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:43.790239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:43.794122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:43.797007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:43.801343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:43.813295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:43.814526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:43.815533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:43.819441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 308/616] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:43.826209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:43.966394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041690-573c-41d3-92ef-366d131d3d4f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.007302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.008059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.008348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.008624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.019058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c7916-fb9e-435f-8563-a8263838f337 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.019329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:44.024795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:44.025747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:44.026599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 309/617] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:44.031866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.100146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.100815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.101097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.101372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.102233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:44.102542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.104001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.114654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:44.137962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad65e6-cd6b-4123-9417-f441b443a059 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.143473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 309/618] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 22:49:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:44.149593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.174760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.175375 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.175681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.175953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.176615 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:44.176920 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.178291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.186923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 310/619] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:44.193116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.226296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:44.275394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de7040-933e-4108-9bf4-7db41d16f037 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.299527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92205ca-e43b-42db-8005-fefe9967b998 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.312511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.313470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.313962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.314435 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.325311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.327031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.335906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.336708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.336829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.337101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:44.343467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:44.344426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:44.345297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:44.345297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:44.345297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:44.347326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 310/620] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:44.353561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.354594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:44.357379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.359590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:44.385457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:44.386509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:44.387394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:44.387394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:44.387394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:44.387394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 311/621] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:44.396559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.467619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ec6ea-8c06-469a-9c96-8bed026c88e0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.477795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:44.505472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.506434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.506927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.507396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.508804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:44.509353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.510971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.524417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 311/622] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:44.530019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.633676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033bb3-4b91-4bcc-b46d-631992ced07d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.641813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fabf4-808f-45aa-8c3f-7c3bcb56202d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.670587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.671631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.672111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.672580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.680636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.682382 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:44.694483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:44.695415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:44.696397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:44.698727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 312/623] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:44.704500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.733224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.733470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.733752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.734018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.734659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:44.734954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.736352 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.741745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:44.750253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 312/624] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:44.755185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.817863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2b9b5-ce87-4089-8a8d-4ebbbfb8c15c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.852429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:44.853304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:44.853801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:44.854292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:44.855223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:44.855782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:44.856230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:44.857339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:44.871092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 313/625] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:44.876793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:44.934042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31473261-78c0-42de-8444-5cbb37f96f71 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:44.967227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:44.980566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9058d-fb4b-4062-96e4-ef54ae0d9f5d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:45.001250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:45.002140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.002618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.003171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.016874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:45.018316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:45.020049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:45.020705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.020979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.021248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/e7ceb38886d34f9d8e59b2871989446c/roles/8d7867fc676d4963a98f9eaed5f9f9f6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:45.042448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:45.043473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:45.044536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 313/626] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:49:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:45.049174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=e7ceb38886d34f9d8e59b2871989446c, role_id=8d7867fc676d4963a98f9eaed5f9f9f6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:45.050627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:45.053611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:45.075585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 314/627] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:49:44 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/e7ceb38886d34f9d8e59b2871989446c/roles/8d7867fc676d4963a98f9eaed5f9f9f6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:49:45.178525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29870116-faf7-4fa0-baef-33477fbd8c7b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:45.196545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:45.218596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d46b15-32d7-4719-bd4f-e9442bc371bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:45.219222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d46b15-32d7-4719-bd4f-e9442bc371bf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.219526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d46b15-32d7-4719-bd4f-e9442bc371bf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.219807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d46b15-32d7-4719-bd4f-e9442bc371bf None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.220608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d46b15-32d7-4719-bd4f-e9442bc371bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:45.220921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d46b15-32d7-4719-bd4f-e9442bc371bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:45.222449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d46b15-32d7-4719-bd4f-e9442bc371bf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:45.247828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 314/628] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:49:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:45.249823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:45.342358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763441a4-1916-4715-b069-3c586e4212c5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.342710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763441a4-1916-4715-b069-3c586e4212c5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.342982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763441a4-1916-4715-b069-3c586e4212c5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.344666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 315/629] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:45.349821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055ca9a-50cd-43df-9baf-509f0ce8d473 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.350106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055ca9a-50cd-43df-9baf-509f0ce8d473 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.350386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055ca9a-50cd-43df-9baf-509f0ce8d473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.374261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7055ca9a-50cd-43df-9baf-509f0ce8d473 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:45.376048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7055ca9a-50cd-43df-9baf-509f0ce8d473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:45.407380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617e989-6428-4529-a596-21b0539d2a29 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:45.460213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:45.462335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 316/630] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:45.463170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.463681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.464201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.479545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:45.530340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e48dc-f9a5-4fc4-8639-bcd7bd5a4b7d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.530678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e48dc-f9a5-4fc4-8639-bcd7bd5a4b7d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.530896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e48dc-f9a5-4fc4-8639-bcd7bd5a4b7d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.532381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 317/631] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:49:45.537622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61914339-d32c-4c95-a6e3-8ce89f54bc8a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:45.537910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61914339-d32c-4c95-a6e3-8ce89f54bc8a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:45.538188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61914339-d32c-4c95-a6e3-8ce89f54bc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:45.561038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:45.562797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:45.576129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-61914339-d32c-4c95-a6e3-8ce89f54bc8a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:45.577318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-61914339-d32c-4c95-a6e3-8ce89f54bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:45.691223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 318/632] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:45.748406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 315/633] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:45.754599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61914339-d32c-4c95-a6e3-8ce89f54bc8a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:45.800707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:45.806031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:45.879829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:45.931010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61914339-d32c-4c95-a6e3-8ce89f54bc8a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:45.961421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420d3ca-742e-4025-ad4d-2bd12709350b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:45.999376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.000037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.000314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.000580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.021775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.023240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.033511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.034144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.034418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.034686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.046819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:46.052042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:46.052990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:46.053830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 316/634] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:46.056335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:46.057241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:46.058120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:46.058120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:46.058120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:46.063913 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 319/635] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:46.069548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.070079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.164170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:46.189584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742b86c-d24f-43df-b25e-7d1c6a59fbc9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.210649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:46.216819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a880a42-aed7-44fe-9170-0227c6dff252 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.229724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.230638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.231151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.231663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.232538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:46.233105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.234683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.248843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 320/636] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:46.249984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.259174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.259900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.260181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.260453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.261120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:46.261421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.263654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.286560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 317/637] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:46.286560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.394578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2df04-c002-437f-99ef-f1c96c5ed29b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.399774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7143dbb-0bb7-480f-81d0-13ae3069ba2f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.431522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4571-98d7-4169-be97-df9492d79e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.432227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4571-98d7-4169-be97-df9492d79e79 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.432507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4571-98d7-4169-be97-df9492d79e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.432783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4571-98d7-4169-be97-df9492d79e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.440682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.441557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.442026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.442363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.442835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a4571-98d7-4169-be97-df9492d79e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.444305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a4571-98d7-4169-be97-df9492d79e79 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.453977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:46.459183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:46.460225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:46.461178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-429a4571-98d7-4169-be97-df9492d79e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:46.462327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:46.463166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:46.464080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:46.464080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:46.464080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:46.464248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 321/638] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:46.464248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.470305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 318/639] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:46.470305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79511b4-69f4-4810-aa3d-fdbae6c1bb76 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.470305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79511b4-69f4-4810-aa3d-fdbae6c1bb76 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.470305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79511b4-69f4-4810-aa3d-fdbae6c1bb76 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.470305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 319/640] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:46.470305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79511b4-69f4-4810-aa3d-fdbae6c1bb76 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.471826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:46.482335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:46.574291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79511b4-69f4-4810-aa3d-fdbae6c1bb76 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.580888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c32b4c-4e41-4b18-93c3-fc98d285c771 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.611113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.611767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.612046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.612346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.613009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:46.613332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.614669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.616785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327d6f1-687d-4ef4-91f0-746b3e1e2501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.617577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d6f1-687d-4ef4-91f0-746b3e1e2501 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.617957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d6f1-687d-4ef4-91f0-746b3e1e2501 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.618327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d6f1-687d-4ef4-91f0-746b3e1e2501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.622863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 320/641] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:46.623467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.630082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327d6f1-687d-4ef4-91f0-746b3e1e2501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.631671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327d6f1-687d-4ef4-91f0-746b3e1e2501 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-8327d6f1-687d-4ef4-91f0-746b3e1e2501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:46.643851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:46.644983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:46.646088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:46.646569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 322/642] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:46.652605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcae516-157c-43b5-a17a-bdf9429e51dd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.652896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcae516-157c-43b5-a17a-bdf9429e51dd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.653180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcae516-157c-43b5-a17a-bdf9429e51dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.672388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1fcae516-157c-43b5-a17a-bdf9429e51dd None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:46.674610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1fcae516-157c-43b5-a17a-bdf9429e51dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:46.718550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:46.745162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86dffd-6a8d-43c7-9d3b-a0456f384400 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.745922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:46.755509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 323/643] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:46.756349 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcae516-157c-43b5-a17a-bdf9429e51dd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.793323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.794908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.795692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.796183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.797283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:46.797800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.800279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.816285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 321/644] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:46.817306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.870057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcae516-157c-43b5-a17a-bdf9429e51dd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.905371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.906164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.906542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.906911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.917647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.919196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:46.934997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:46.936221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:46.937494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:46.937494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:46.937494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:46.942717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492655a-e5c6-43ac-b78f-ffc3282e78f0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:46.942897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 324/645] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:46.942897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:46.982328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:46.990058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:46.990850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:46.991230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:46.991634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:46.992963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:46.993369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:46.995090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.002259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:47.010153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 322/646] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:47.012096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.072358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf056a7-81fa-492b-bd75-7c9eb30fc1e9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.110661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.111291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.111593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.111865 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.119417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.120877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:47.131952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:47.132917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:49:47.133875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:47.134455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c75fc-69cf-4a0e-ac71-43cdf60684ea None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.136172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 325/647] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:47.136443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.170499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.171105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.171384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.171684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.172441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:47.172869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.174235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.192258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 323/648] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:47.192258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.242905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca260-5fb7-44ff-b04b-7cfcf8528e62 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.243875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:47.250918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:47.282311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.283308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.284023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.284855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.286028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=786e4e7b1c5147e093693767b229e52b) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:47.286595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.288319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.293638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc075c-8c25-4d11-b7dc-a97a9f201298 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.297007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 326/649] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/role_assignments?user.id=786e4e7b1c5147e093693767b229e52b&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:47.297965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.344066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.345152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.345533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.345922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.363655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:47.370527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:47.371892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:47.372776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:47.375699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 324/650] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:47.377001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.454931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a742e36-23a3-4755-9f05-cdb1025ce98c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.497128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.497661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.497936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.498203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.498918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:47.499223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.500678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.502549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6beff-2a87-4dd9-b7a7-6af433fe176e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.518345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:47.519863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 327/651] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:47.520067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.542620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82407c-dc61-4c7f-973e-da6f41213142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.543249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82407c-dc61-4c7f-973e-da6f41213142 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.543943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82407c-dc61-4c7f-973e-da6f41213142 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.543943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82407c-dc61-4c7f-973e-da6f41213142 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.552307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82407c-dc61-4c7f-973e-da6f41213142 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.553757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82407c-dc61-4c7f-973e-da6f41213142 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.557403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7c82407c-dc61-4c7f-973e-da6f41213142 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:47.568262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:47.569415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:47.570730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 325/652] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:47.571556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca40f-3710-4c5b-b707-8b6df72aec9a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.571556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca40f-3710-4c5b-b707-8b6df72aec9a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.571556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca40f-3710-4c5b-b707-8b6df72aec9a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.573575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 326/653] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:47.573575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca40f-3710-4c5b-b707-8b6df72aec9a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.637461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5cdb-9094-4f07-8f90-89538a5b83a4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.673940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.674631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.674812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.675078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.676011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:47.676011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.677379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.683242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca40f-3710-4c5b-b707-8b6df72aec9a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.686241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 328/654] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:47.686692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.730090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.730765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.731048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.731322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.775011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:47.789892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:47.801367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f8e02-e260-4a79-a54c-11d694fc575f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:47.805310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.806782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.838993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781eab6-0c2d-41ad-bd52-d2ea2b618b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:47.840062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781eab6-0c2d-41ad-bd52-d2ea2b618b3a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.840483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781eab6-0c2d-41ad-bd52-d2ea2b618b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.840899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781eab6-0c2d-41ad-bd52-d2ea2b618b3a None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.841789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781eab6-0c2d-41ad-bd52-d2ea2b618b3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:47.842363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781eab6-0c2d-41ad-bd52-d2ea2b618b3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:47.844087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781eab6-0c2d-41ad-bd52-d2ea2b618b3a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:47.863257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 329/655] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:47.864535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a7a86-d59d-4447-ad81-342ce01fbd96 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:47.864822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a7a86-d59d-4447-ad81-342ce01fbd96 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:47.865098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a7a86-d59d-4447-ad81-342ce01fbd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:47.884070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e57a7a86-d59d-4447-ad81-342ce01fbd96 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:47.885271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e57a7a86-d59d-4447-ad81-342ce01fbd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:47.924039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 327/656] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:47.924802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:47.960281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 330/657] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:47.961198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a7a86-d59d-4447-ad81-342ce01fbd96 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.003018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.040813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b806b0-5cca-4e57-afa9-a1ffc0dcdd04 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.076179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.076838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.077117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.077394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.078125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.088122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.089839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:48.106419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:48.107445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:48.108379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:48.108379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:48.108379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:48.110650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a7a86-d59d-4447-ad81-342ce01fbd96 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.111867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 328/658] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:48.112711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.159049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.160299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.161284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.161969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.236346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.236543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e331-56e5-47ed-80d5-c50ac44d65fe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.237803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.265271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.277830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.278759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.279042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.279313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.290423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.291986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:48.310473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:48.312208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:48.313369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:48.313369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:48.313369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:48.313369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 329/659] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:48.313369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.333286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.361384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 331/660] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:48.361384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.421540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7369aa-ab2f-45ca-9e64-8a2d51f9f8f4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.456531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.457134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.457413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.457698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.458318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:48.458614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.460010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.464606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c697cf-c546-4d42-9905-2be7e6860fc2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.472480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 330/661] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:48.473282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.504012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.504648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.504922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.505195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.508665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.515960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:48.520810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:48.521724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:48.522567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 332/662] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:48.523258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.563829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.589605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d28da-f6fb-4d39-a4b2-af1959132240 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.638504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.639112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.639391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.639690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.640323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:48.640625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.642026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.643736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985910-b3d3-46b0-ac94-118651b26bcd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.653792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 331/663] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:48.654587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.686006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.687049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.687680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.688273 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.699645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.701075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:48.716667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:48.717613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:48.719186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:48.719186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:48.719186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:48.719186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 333/664] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:48.719186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.766696 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.775617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e12c4-922c-4434-8772-7ab6e1c9fcc5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.810876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.811516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.811816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.812081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.818495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:48.821092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.822518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.828728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bf45b-488a-4c5a-a03a-f0c5ff723d63 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:48.835109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:48.836133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:48.837081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 332/665] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:48.837683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.870043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.870650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.871067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.871341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.872574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:48.872883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.874246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:48.887565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 334/666] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:48.887565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:48.949966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee85516-3526-4c99-ab24-bfa4cf1e8e9a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:48.985598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:48.986234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:48.986513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:48.986786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:48.994042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:48.995251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:49.006531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:49.007395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 333/667] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:49.009436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.018144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d52ff-e03e-467f-9192-61d48fb2a76e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.027724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.054060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.054689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.055458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.055841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.056380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:49.056680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.058020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.070426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 335/668] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:49.071591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.088969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.115363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29f59c-01c0-44ee-90ae-428025da9d8c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.153978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.154590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.154865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.155134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.155879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:49.156179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.157596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.175420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 334/669] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:49.176453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.193916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c259d2-0fbb-49d4-9929-84a3cc0eb020 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.232624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.233553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.233835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.234160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.247262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.248771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.253959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:49.265062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:49.266189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:49.267237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:49.267237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:49.267237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:49.267237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 336/670] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:49.267237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.286022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfa02-a757-4af8-8991-d233e4b2be56 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.324730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.325531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.325811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.326079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.326924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:49.327562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.328998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.333842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.345629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 335/671] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:49.346326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.375733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20073f-21c0-4188-80f5-626615871867 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.411786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.412416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.412692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.412960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.422534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.424422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:49.438921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:49.440146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:49.441032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:49.441470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 337/672] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:49.442049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.454577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b96647-abbc-49ec-b96e-ea376b5acfda None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.491913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.493580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.493866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.494465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.497399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.555923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61681c0e-03a5-4e40-acc2-1ae1a2bfb2f1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.564566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.566047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.580214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.591286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.591924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.592205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.592478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.593112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:49.593416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.594781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.607058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 338/673] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:49.607872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.670646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 336/674] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:49.671568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.704150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.707388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fe004-cdd8-441e-96bc-39c518afb2da None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.740826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41b177-eec5-428e-b7dd-934b44a587f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.741429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41b177-eec5-428e-b7dd-934b44a587f6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.741734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41b177-eec5-428e-b7dd-934b44a587f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.742003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41b177-eec5-428e-b7dd-934b44a587f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.743066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41b177-eec5-428e-b7dd-934b44a587f6 None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=56228e09c8b742b29e49c40d1a39870f) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:49.743371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41b177-eec5-428e-b7dd-934b44a587f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.744806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41b177-eec5-428e-b7dd-934b44a587f6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.750432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 339/675] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/role_assignments?user.id=56228e09c8b742b29e49c40d1a39870f&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:49.751822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b66b4-7052-49b1-883c-4d3eb204df2d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.752099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b66b4-7052-49b1-883c-4d3eb204df2d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.752738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b66b4-7052-49b1-883c-4d3eb204df2d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.754079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 340/676] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:49.754709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b66b4-7052-49b1-883c-4d3eb204df2d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.784897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2294480-44b8-4d40-b1e8-a3a7f839ea2f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.814655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.826029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.826663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.826951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.827224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.827957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:49.828265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.829740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.842263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 337/677] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:49.843111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:49.879281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b66b4-7052-49b1-883c-4d3eb204df2d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.916904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abacba-33b9-4006-a31c-373246ef6a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.917547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abacba-33b9-4006-a31c-373246ef6a25 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.917822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abacba-33b9-4006-a31c-373246ef6a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.918086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abacba-33b9-4006-a31c-373246ef6a25 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.927264 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abacba-33b9-4006-a31c-373246ef6a25 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:49.928750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abacba-33b9-4006-a31c-373246ef6a25 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d9abacba-33b9-4006-a31c-373246ef6a25 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:49.941362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:49.942337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:49.943271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 341/678] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:49.943960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a9e1c-9dbd-4907-ae99-6ba377c9dd2d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.944070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a9e1c-9dbd-4907-ae99-6ba377c9dd2d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.944343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a9e1c-9dbd-4907-ae99-6ba377c9dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:49.950111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:49.958907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa5310-a4ab-4556-afb7-cb16087be062 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:49.963704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-db1a9e1c-9dbd-4907-ae99-6ba377c9dd2d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:49.964905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-db1a9e1c-9dbd-4907-ae99-6ba377c9dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:49.998042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:49.998676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:49.998953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:49.999222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.010567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:50.012062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:50.026811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:50.027790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:50.028682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:50.028682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:50.028682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:50.028682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 338/679] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:50.028901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.043531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 342/680] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:50.043531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a9e1c-9dbd-4907-ae99-6ba377c9dd2d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.094802 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:50.144526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27d8ef-76b3-4daa-93f4-c30f58ffaf93 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.181841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a9e1c-9dbd-4907-ae99-6ba377c9dd2d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.187698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:50.188470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:50.188919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:50.189366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.190220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=6cf808a1f5dd45bea0a3a81131532b2b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:50.191172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:50.192802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:50.201522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 339/681] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/role_assignments?user.id=6cf808a1f5dd45bea0a3a81131532b2b&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:50.203173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.220416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:50.224402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:50.227216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:50.227648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:50.228041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.228963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:50.229409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:50.231205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:50.267231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 343/682] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:50.274456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.368702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea39f3-2298-4c31-97eb-f44a15eed437 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.408394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:50.408394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:50.408394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:50.408756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.409294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:50.411264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:50.411264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:50.416750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:50.422835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c2f19-1c26-4c35-9e39-e9daf1f057b8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.425590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 340/683] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:50.426497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.469835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:50.471288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:50.471602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:50.471876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.515862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:50.552117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d324288-4bc6-4a02-9373-230f9a9c79dc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.556002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:50.557611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:50.597671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:50.598734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:50.599188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:50.599675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.613697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:50.620432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:50.621338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:50.622216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:50.625031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 341/684] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:50.626301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.662915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:50.673100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 344/685] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 22:49:50.673600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.763142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151547f-80e0-4d68-bfa3-30d239b41f55 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.797732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4dcbd-fdcf-43b1-b845-83b83799a79f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.805841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:50.814186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:50.814904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:50.815761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:50.816124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.825014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:50.827538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:50.838353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:50.839202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:50.839711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:50.840157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:50.840995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:50.841466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:50.842016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:50.842859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:50.843941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 342/686] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:50.844612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.846334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:50.856949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 345/687] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:50.858092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:50.938622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:50.957806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f317f-44b1-487f-83ed-5abe189e7ccc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:50.991506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c276e2-d355-46a2-90bc-2349f1d3d4ed None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.002337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.002986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.003267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.003566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.018776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.020526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.030838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.031816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.032308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.032772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.033644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:51.034156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.035850 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:51.037933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:51.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:51.039837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:51.039837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:51.039837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:51.039837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 343/688] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:51.039837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.053021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 346/689] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:51.054320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.086299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:51.147393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9e395-62d5-4b3c-b620-31417d81c187 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.175364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61874bd6-181c-4d76-896b-a7251bd395bc None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.191215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.192605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.193297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.193772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.196175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:51.210176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.211571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.219441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.220849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.221591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.222064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.222890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:51.223385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.224950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:51.227181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:51.228126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:51.229127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:51.229127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:51.229127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:51.229127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 344/690] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:51.229127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.244441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 347/691] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:51.246555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.334361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:51.349965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02198-1441-457e-833c-9f12cfd4aed9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.370709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db58fd3-53ed-462b-b193-e24b7164f77f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.395579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.397078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.397945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.399186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.400336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=c0e10b1ef7454167bd1c606ef4f70bf3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:51.400838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.403068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.406156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.406984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.407470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.408126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.408966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:51.409404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 345/692] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/role_assignments?user.id=c0e10b1ef7454167bd1c606ef4f70bf3&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:51.409685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.410190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.411831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.434758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 348/693] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:51.447650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.489612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:51.564083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7cd4c-e982-4e18-9357-f5d634862238 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.614790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.615910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.616537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.617029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.627637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.629375 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.637175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1682-c61b-42f5-94a7-516bf9726b3f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:51.642365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:51.643303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:51.644257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:51.647308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 349/694] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:51.654041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.687374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.688127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.688640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.688919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.699175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.701502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:51.717311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:51.718311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:51.719240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 346/695] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:51.725011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.749943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:51.784211 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:51.788466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e974e1-00f9-4022-86f0-f4aa9e985a5a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.827709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.828689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.829204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.829700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.830634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:51.831139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.832789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.858959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 350/696] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:51.860054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:51.872597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89918c-213d-49d2-9052-596a538d3136 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:51.949037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:51.949721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:51.949939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:51.950208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:51.950917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:51.951219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:51.952748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:51.988768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 347/697] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:51.990912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:52.038722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae8ce-91f2-43b0-9e67-e22b8a4be4a6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:52.083698 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:52.084317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.084596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.084855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.103059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:52.119885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:52.134033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdfe8f-32ba-4a66-b911-347f1635a13b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:52.169482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6eea81-a76a-40d4-9b20-ae33b31273dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:52.170350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eea81-a76a-40d4-9b20-ae33b31273dd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.170624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eea81-a76a-40d4-9b20-ae33b31273dd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.170890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eea81-a76a-40d4-9b20-ae33b31273dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.171630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eea81-a76a-40d4-9b20-ae33b31273dd None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=e7ceb38886d34f9d8e59b2871989446c) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:52.171944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eea81-a76a-40d4-9b20-ae33b31273dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:52.173621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eea81-a76a-40d4-9b20-ae33b31273dd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:52.175930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:52.177672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:52.180314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 348/698] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/role_assignments?user.id=e7ceb38886d34f9d8e59b2871989446c&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:52.182093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e591d-f864-44b7-8010-e264477d3e9c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.182093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e591d-f864-44b7-8010-e264477d3e9c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.182295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e591d-f864-44b7-8010-e264477d3e9c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.188138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 349/699] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:52.189821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac45ff7-8bab-44a1-95d7-cf49cf28a153 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.190112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac45ff7-8bab-44a1-95d7-cf49cf28a153 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.190384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac45ff7-8bab-44a1-95d7-cf49cf28a153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.208916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ac45ff7-8bab-44a1-95d7-cf49cf28a153 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:52.210244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ac45ff7-8bab-44a1-95d7-cf49cf28a153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:52.303025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 351/700] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:52.308646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 350/701] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:52.310054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:52.373476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:52.422726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ffdb7-fad6-4ddb-a5a4-4c25abd6f84d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:52.466176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:52.467627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.468114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.468594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.480479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:52.482185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:52.500413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:52.501495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:52.502376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:52.502376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:52.502376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:52.504527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 352/702] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:52.510911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac45ff7-8bab-44a1-95d7-cf49cf28a153 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:52.605773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:52.642214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:52.658990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac45ff7-8bab-44a1-95d7-cf49cf28a153 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:52.707677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:52.708581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.708860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.709583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.710251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:52.710553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:52.713209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:52.731017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 351/703] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:52.736698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:52.829709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5a43-c12e-4ca2-8374-17a55f8d09b5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:52.844726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:52.859268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052532ae-8ed4-4add-b9d1-f1eb358ff4ab None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:52.873992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa17f9f-95bb-44eb-b04c-3648250f2480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:52.874951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa17f9f-95bb-44eb-b04c-3648250f2480 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.875575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa17f9f-95bb-44eb-b04c-3648250f2480 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.876224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa17f9f-95bb-44eb-b04c-3648250f2480 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.877005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa17f9f-95bb-44eb-b04c-3648250f2480 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:52.878648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa17f9f-95bb-44eb-b04c-3648250f2480 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2aa17f9f-95bb-44eb-b04c-3648250f2480 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:52.890941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:52.892347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:49:52.893395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:52.894940 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:52.897249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 353/704] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 9 22:49:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:52.900149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be8d4-02e6-4d4a-b688-14102fbcc0de None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.900570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be8d4-02e6-4d4a-b688-14102fbcc0de None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.900973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be8d4-02e6-4d4a-b688-14102fbcc0de None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.906589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dc58b-2a4c-4068-8e0f-e12ebff13b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:52.906589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dc58b-2a4c-4068-8e0f-e12ebff13b6e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.906589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dc58b-2a4c-4068-8e0f-e12ebff13b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.906589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dc58b-2a4c-4068-8e0f-e12ebff13b6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.907167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 354/705] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:52.908258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be8d4-02e6-4d4a-b688-14102fbcc0de None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:52.914851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dc58b-2a4c-4068-8e0f-e12ebff13b6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:52.916693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dc58b-2a4c-4068-8e0f-e12ebff13b6e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d75dc58b-2a4c-4068-8e0f-e12ebff13b6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:52.933171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:52.934126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:52.935029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:52.941967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 352/706] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:52.943614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabbcfe-6e91-4b94-b10f-dcf1fd10cd60 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:52.944167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabbcfe-6e91-4b94-b10f-dcf1fd10cd60 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:52.944662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabbcfe-6e91-4b94-b10f-dcf1fd10cd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:52.964468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7fabbcfe-6e91-4b94-b10f-dcf1fd10cd60 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:52.966043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7fabbcfe-6e91-4b94-b10f-dcf1fd10cd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:53.040836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be8d4-02e6-4d4a-b688-14102fbcc0de None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:53.061607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 353/707] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:53.062954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabbcfe-6e91-4b94-b10f-dcf1fd10cd60 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:53.089690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:53.090311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:53.090593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:53.090867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:53.091523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:53.091832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:53.093212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:53.103164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 355/708] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 22:49:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:53.112511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:53.177259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:53.208485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabbcfe-6e91-4b94-b10f-dcf1fd10cd60 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:53.235859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:53.262598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:53.263224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:53.263519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:53.264056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:53.266739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:53.267053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:53.269366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:53.286300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 354/709] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:53.294607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:53.297229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e67e2-5872-443e-a5da-5320d5ec43a1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:53.358241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:53.358857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:53.359137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:53.359409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:53.360064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:53.360368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:53.361741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:53.368931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 356/710] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 22:49:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:53.399882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:53.486081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb40da-6fd1-41d1-93a8-86280684aadc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:53.561228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:53.562531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:53.562975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:53.563389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:53.564316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=632d504031cb466299331db2d9184912, user.id=786e4e7b1c5147e093693767b229e52b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:53.564771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:53.566367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:53.577313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 355/711] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:53 2022] GET /identity/v3/role_assignments?user.id=786e4e7b1c5147e093693767b229e52b&scope.project.id=632d504031cb466299331db2d9184912&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:53.581978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:53.591574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41a6dc-da7f-40d7-b491-6a3fe6a7c56d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:53.638848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:53.639791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:53.640218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:53.640660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:53.654743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:53.656497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:53.673812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:53.674844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:53.676244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:53.676244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:53.676244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:53.676244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 357/712] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:49:53.684948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:53.745263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:53.808717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af07622-4c1e-4f93-9a22-b5e3c9626758 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:53.846020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc307c-43f8-4282-80a1-6e0780dee194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:53.847526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc307c-43f8-4282-80a1-6e0780dee194 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:53.848145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc307c-43f8-4282-80a1-6e0780dee194 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:53.848625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc307c-43f8-4282-80a1-6e0780dee194 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:53.849505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc307c-43f8-4282-80a1-6e0780dee194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:53.850059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc307c-43f8-4282-80a1-6e0780dee194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:53.852240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc307c-43f8-4282-80a1-6e0780dee194 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:53.865355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 356/713] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:53.871716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:53.980680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f9f50-5a35-42df-b3f3-163d8af3873e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:54.020734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbc13a-1527-4e1e-b418-3bfb1b2e7747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:54.021787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbc13a-1527-4e1e-b418-3bfb1b2e7747 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:54.022279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbc13a-1527-4e1e-b418-3bfb1b2e7747 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:54.022758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbc13a-1527-4e1e-b418-3bfb1b2e7747 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:54.023738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbc13a-1527-4e1e-b418-3bfb1b2e7747 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:54.025397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbc13a-1527-4e1e-b418-3bfb1b2e7747 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:54.058580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 358/714] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:49:53 2022] POST /identity/v3/users => generated 254 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:49:54.213779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713698-a192-49bd-89d4-eef505041aad None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:54.214444 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713698-a192-49bd-89d4-eef505041aad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:54.214948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713698-a192-49bd-89d4-eef505041aad None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:54.216855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 357/715] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:54.222602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a4ae-733f-46b5-8e66-96bd7a9dcf89 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:54.223171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a4ae-733f-46b5-8e66-96bd7a9dcf89 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:54.223740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a4ae-733f-46b5-8e66-96bd7a9dcf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:54.244223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e820a4ae-733f-46b5-8e66-96bd7a9dcf89 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:54.245825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e820a4ae-733f-46b5-8e66-96bd7a9dcf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:54.248392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:54.325097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 359/716] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:54.602627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713698-a192-49bd-89d4-eef505041aad None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:54.711392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713698-a192-49bd-89d4-eef505041aad None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:54.721240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:54.748918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:54.749575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:54.749859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:54.750129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] PATH_INFO: `/v3/services/network` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:54.750724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:54.752153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:54.764418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:54.765502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:54.766360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 358/717] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:54 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:54.771705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820a4ae-733f-46b5-8e66-96bd7a9dcf89 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:54.874549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820a4ae-733f-46b5-8e66-96bd7a9dcf89 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:54.909687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c2f62-8b1d-4921-b304-041bde0f97ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:54.910309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c2f62-8b1d-4921-b304-041bde0f97ce None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:54.910591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c2f62-8b1d-4921-b304-041bde0f97ce None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:54.910866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c2f62-8b1d-4921-b304-041bde0f97ce None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:54.911512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c2f62-8b1d-4921-b304-041bde0f97ce None admin] RBAC: Adding query filter params (name=network) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:54.911822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c2f62-8b1d-4921-b304-041bde0f97ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:54.913188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c2f62-8b1d-4921-b304-041bde0f97ce None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:54.918410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 360/718] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:49:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:54.924255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:55.088208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d776a9-ca1a-4cad-a15c-c97c2f10f00b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:55.115437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68a464-415a-4509-9cb2-994de3e5a685 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:55.115798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68a464-415a-4509-9cb2-994de3e5a685 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:55.116061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68a464-415a-4509-9cb2-994de3e5a685 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:55.118200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 361/719] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:55.124192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6cd3f-0da3-4ccc-80cd-0ed9ae43a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:55.124479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6cd3f-0da3-4ccc-80cd-0ed9ae43a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:55.124810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6cd3f-0da3-4ccc-80cd-0ed9ae43a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:55.131830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:55.132491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:55.133992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:55.134392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:55.134569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:55.135262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] RBAC: Adding query filter params (type=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:55.135681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:55.137058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:55.143532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 359/720] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:49:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:55.153762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9ab6cd3f-0da3-4ccc-80cd-0ed9ae43a1c1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:55.155051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9ab6cd3f-0da3-4ccc-80cd-0ed9ae43a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:55.254218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 362/721] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:55.621988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:55.661459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:55.734398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02700bad-6ad2-4bf8-b5c0-b7707214257a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:55.769922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:55.770560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:55.770838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:55.771109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:55.782377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:55.787772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:55.788728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:55.790384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 360/722] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:55.796608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6cd3f-0da3-4ccc-80cd-0ed9ae43a1c1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:55.927356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6cd3f-0da3-4ccc-80cd-0ed9ae43a1c1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:55.965844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:55.970879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f7c9d-c405-4e49-9168-c2b6b3e0e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:55.971656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f7c9d-c405-4e49-9168-c2b6b3e0e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:55.971938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f7c9d-c405-4e49-9168-c2b6b3e0e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:55.972218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f7c9d-c405-4e49-9168-c2b6b3e0e9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:55.972883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f7c9d-c405-4e49-9168-c2b6b3e0e9c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:55.973186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f7c9d-c405-4e49-9168-c2b6b3e0e9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:55.974726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f7c9d-c405-4e49-9168-c2b6b3e0e9c3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:55.984161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 363/723] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:55.989998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:56.094572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:56.101611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5c17-bf50-49eb-a5b2-1e143c0fc62f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:56.137643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:56.138283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.138565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.138840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.150136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:56.151620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:56.166683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:56.167785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:56.168680 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:56.168680 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:56.168680 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:56.168680 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 361/724] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:56.172880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763472a1-46cb-41a6-9fda-53f759167f36 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.173406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763472a1-46cb-41a6-9fda-53f759167f36 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.173911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:56.174032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763472a1-46cb-41a6-9fda-53f759167f36 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.175808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 364/725] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:56.181224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc25248-87b8-4173-9d4e-968cbb82cfd8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.181762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc25248-87b8-4173-9d4e-968cbb82cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.182253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc25248-87b8-4173-9d4e-968cbb82cfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.224574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-afc25248-87b8-4173-9d4e-968cbb82cfd8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:56.226176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-afc25248-87b8-4173-9d4e-968cbb82cfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:56.286822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96108cc-da3c-4280-a270-daa6cf91c4b5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:56.323445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:56.324081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.324360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.324631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.325261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:56.325589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:56.326931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:56.339558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 362/726] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:56.345034 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:56.354368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:56.382403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 365/727] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:56.396840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e725-6d9f-411c-a657-87ef33f3679f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.397180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e725-6d9f-411c-a657-87ef33f3679f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e725-6d9f-411c-a657-87ef33f3679f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.399290 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 366/728] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:56.406258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff19f9-48c9-4f14-a102-b314de68e37b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.406609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff19f9-48c9-4f14-a102-b314de68e37b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.406885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff19f9-48c9-4f14-a102-b314de68e37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.433529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e8ff19f9-48c9-4f14-a102-b314de68e37b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:56.434703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e8ff19f9-48c9-4f14-a102-b314de68e37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:56.452411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f42ce-042e-48a1-86a4-207d5ebada84 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:56.487651 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:56.488656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.489142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.489622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.498462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:56.500361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:56.505694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 367/729] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:56.514085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:56.515063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:49:56.516070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:56.518569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 363/730] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:56.527057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff19f9-48c9-4f14-a102-b314de68e37b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:56.608326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:56.656239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff19f9-48c9-4f14-a102-b314de68e37b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:56.691309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:56.691884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.692168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.692445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.693170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:56.693477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:56.694902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:56.714028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 368/731] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 22:49:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:56.720583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:56.791219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:56.819862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:56.876445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a15f36-d45e-44be-b8f7-e444f169f4be None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:56.911440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:56.912098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.912378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.912647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.940383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42033135-ffec-46cd-81aa-8e8b36092c72 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:56.955959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:56.978305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:56.978968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:56.979257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:56.979567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:56.980242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:56.981607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:56.993655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:56.995516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:57.003815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 369/732] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:49:56 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:49:57.005132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.102295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:57.175658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 364/733] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 22:49:57.181091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.220585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3cd6c-e1fe-4624-8aa8-a18903c973aa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:57.274466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:57.275760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.276045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.276602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.290498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:57.297371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:57.298310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:49:57.299148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:57.299835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 370/734] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:57.306138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.324447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a82dd-55a5-4206-b600-1986e7d94e32 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:57.343312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:57.374174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:57.374814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.375095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.375368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.389185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:57.390765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:57.409970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:57.410959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:57.412570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:57.412570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:57.412570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:57.412570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 365/735] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:57.417542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.444004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:57.456717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd726102-eedb-48a6-8584-78180133c566 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:57.502605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8684cbd-02b4-4a03-a7f7-9e4b88dc0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:57.503135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8684cbd-02b4-4a03-a7f7-9e4b88dc0f4b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.503409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8684cbd-02b4-4a03-a7f7-9e4b88dc0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.503722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8684cbd-02b4-4a03-a7f7-9e4b88dc0f4b None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.504378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8684cbd-02b4-4a03-a7f7-9e4b88dc0f4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:57.504678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8684cbd-02b4-4a03-a7f7-9e4b88dc0f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:57.506047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8684cbd-02b4-4a03-a7f7-9e4b88dc0f4b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:57.516061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 371/736] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:57.517393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca660cf1-5995-474b-9fc4-6e0671631d07 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.517683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca660cf1-5995-474b-9fc4-6e0671631d07 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.517954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca660cf1-5995-474b-9fc4-6e0671631d07 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.519221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 372/737] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:57.524447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca660cf1-5995-474b-9fc4-6e0671631d07 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.548810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a7b90-4920-4a30-8d93-e1b46c01bc38 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:57.584194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90540-6018-49c5-a8d5-4e208ae8c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:57.584805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90540-6018-49c5-a8d5-4e208ae8c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.585081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90540-6018-49c5-a8d5-4e208ae8c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.585347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90540-6018-49c5-a8d5-4e208ae8c5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.586776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90540-6018-49c5-a8d5-4e208ae8c5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:57.587292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90540-6018-49c5-a8d5-4e208ae8c5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:57.589515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90540-6018-49c5-a8d5-4e208ae8c5a6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:57.602328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 366/738] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:57.603801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8d9b-9d07-47be-a031-75aa63ddc000 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.604071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8d9b-9d07-47be-a031-75aa63ddc000 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.604336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8d9b-9d07-47be-a031-75aa63ddc000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.623728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e22b8d9b-9d07-47be-a031-75aa63ddc000 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:57.625017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e22b8d9b-9d07-47be-a031-75aa63ddc000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:57.637846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:57.650933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca660cf1-5995-474b-9fc4-6e0671631d07 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:57.688788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:57.689665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.689665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.689964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.701476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:57.703089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:57.723679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:57.724700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:57.725627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:57.725627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:57.725627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:57.726844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 367/740] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:57.728362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8d9b-9d07-47be-a031-75aa63ddc000 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.736087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 373/740] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:57.736087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.758096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:57.899178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b5804-1aba-45e5-a887-94f1f1a144fc None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:57.938904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8d9b-9d07-47be-a031-75aa63ddc000 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:57.946260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d88f0-1761-4d86-a8cc-c32c9c771325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:57.946878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d88f0-1761-4d86-a8cc-c32c9c771325 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.947144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d88f0-1761-4d86-a8cc-c32c9c771325 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.947412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d88f0-1761-4d86-a8cc-c32c9c771325 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.948084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d88f0-1761-4d86-a8cc-c32c9c771325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:57.948381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d88f0-1761-4d86-a8cc-c32c9c771325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:57.949755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d88f0-1761-4d86-a8cc-c32c9c771325 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:57.963693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 374/741] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:57.964959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e79d0e-0f92-439e-b001-f9966786189e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.965240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e79d0e-0f92-439e-b001-f9966786189e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.965512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e79d0e-0f92-439e-b001-f9966786189e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.966925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 375/742] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:57.971562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e79d0e-0f92-439e-b001-f9966786189e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:57.977221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:57.987258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb7d51-0dbc-4731-b199-07f2df0290d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:57.988257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb7d51-0dbc-4731-b199-07f2df0290d1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:57.988696 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb7d51-0dbc-4731-b199-07f2df0290d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:57.989109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb7d51-0dbc-4731-b199-07f2df0290d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:57.996757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb7d51-0dbc-4731-b199-07f2df0290d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:57.998567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb7d51-0dbc-4731-b199-07f2df0290d1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6dcb7d51-0dbc-4731-b199-07f2df0290d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:58.011592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:58.012632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:58.013734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:58.015840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 368/743] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:58.017257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca0c79-2780-402c-bfb2-1a99142e2bba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.017717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca0c79-2780-402c-bfb2-1a99142e2bba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.018139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca0c79-2780-402c-bfb2-1a99142e2bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.037300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-aaca0c79-2780-402c-bfb2-1a99142e2bba None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:58.038825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aaca0c79-2780-402c-bfb2-1a99142e2bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:58.088790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e79d0e-0f92-439e-b001-f9966786189e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.101806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:58.107545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 369/744] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 22:49:58.108446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca0c79-2780-402c-bfb2-1a99142e2bba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.137654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.138783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.139295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.139769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.147568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.149561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:58.162625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:58.163728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:58.164685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 376/745] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:49:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:58.165296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.239429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca0c79-2780-402c-bfb2-1a99142e2bba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.241490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:58.282575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.283118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.283398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.283704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.284558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:58.284865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.286938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.293565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2219d-9d22-499b-8855-810a61751e9c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.300663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 370/746] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:58.301840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.336734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.337411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.337746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.338049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.349165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:49:58.354906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:58.356041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 377/747] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:58.357288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.365890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:58.409368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b8b7f-c3cf-4b28-abc2-af431926e3a9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.444737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.445861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.446140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.446410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.447168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:58.447468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.448910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.466445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 371/748] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:58.467244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.473182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd66b0-5269-4f22-a4f0-0dc841c11868 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.503704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:58.518962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.519627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.520031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.520257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.520980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=632d504031cb466299331db2d9184912, user.id=e7ceb38886d34f9d8e59b2871989446c) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:58.521288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.522860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.529715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 378/749] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/role_assignments?user.id=e7ceb38886d34f9d8e59b2871989446c&scope.project.id=632d504031cb466299331db2d9184912&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:58.530482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.599264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36c269-8bcc-499c-8866-092fb0f02df6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.619688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:58.641987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.643122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.643671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.644164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.645144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:58.645647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.647287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.662971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 372/750] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:58.664619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.669429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9851c-bc02-4c73-a3b5-79dee120b4e4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.712602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.713221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.713511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.714276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.714531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.715954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.739678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 379/751] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:49:58 2022] POST /identity/v3/services => generated 240 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:49:58.739678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.797486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:58.824122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9098bb-f28a-4060-acb1-5dac54b8de62 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.868970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461807db-e0e3-4d5b-a929-6cc54fe35a0c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:58.900234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.901291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.901770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.902309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.908228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:58.908877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:58.909156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:58.909426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:58.923216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.925276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.932750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:58.934586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:58.942067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:58.943081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:58.943979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:58.943979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:58.943979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:58.951511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 380/752] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:58.951511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:49:58.971517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:58.972520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:49:58.973386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:49:58.973386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:49:58.973386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:58.973386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 373/753] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:58.976616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:59.049334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:59.069216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:59.079197 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd63e00-ad3c-4eac-b852-750fc4dc9f32 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.118879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:59.119577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.119824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.120097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.120744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:59.121045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:59.122436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:59.136217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 381/754] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:49:59.141734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:59.149350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cade268-2ac6-40f4-8071-b2be314acf70 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.194473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:59.195892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.196173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.196444 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.197347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:59.198042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:59.199427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:59.216226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 374/755] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:59.224504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:59.261318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd039a7-75f8-485a-9dd9-5d58326b4c14 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.304230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:59.304952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.305229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.305499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.316984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:59.318436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:49:59.331190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:59.333394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:49:59.335662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 382/756] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:49:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:59.342576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:59.355400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:59.379470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:59.398661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f03254-cdef-488f-b702-c20c74f2beaf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.461414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:59.462059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.462337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.464746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.476835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c89a9-3a86-4df9-b19e-5571589a2a6b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.479864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:59.482642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:49:59.497180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:49:59.499238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:49:59.501426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 375/757] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:49:59.508163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:59.535309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf312ace-86c5-44ab-b5ec-c244d2bf69ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:59.536059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312ace-86c5-44ab-b5ec-c244d2bf69ab None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.536431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312ace-86c5-44ab-b5ec-c244d2bf69ab None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.536715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312ace-86c5-44ab-b5ec-c244d2bf69ab None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.537441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf312ace-86c5-44ab-b5ec-c244d2bf69ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:59.537746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf312ace-86c5-44ab-b5ec-c244d2bf69ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:59.539455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf312ace-86c5-44ab-b5ec-c244d2bf69ab None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:59.564261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 383/758] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 22:49:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:59.565569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55910eaf-d58e-476c-8c71-b0b8b818cd11 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.565861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55910eaf-d58e-476c-8c71-b0b8b818cd11 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.566152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55910eaf-d58e-476c-8c71-b0b8b818cd11 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.567691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 384/759] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:49:59.568467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55910eaf-d58e-476c-8c71-b0b8b818cd11 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:59.635854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132107dd-917f-4eb2-8fc2-f9f84fb6dcd9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.653326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:59.672898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bc0e5-a3a9-409e-b9d4-8a4d3dcbae9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:59.673979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bc0e5-a3a9-409e-b9d4-8a4d3dcbae9d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.674721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bc0e5-a3a9-409e-b9d4-8a4d3dcbae9d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.675205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bc0e5-a3a9-409e-b9d4-8a4d3dcbae9d None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.676156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bc0e5-a3a9-409e-b9d4-8a4d3dcbae9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:49:59.676698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bc0e5-a3a9-409e-b9d4-8a4d3dcbae9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:59.679000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bc0e5-a3a9-409e-b9d4-8a4d3dcbae9d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:59.693624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55910eaf-d58e-476c-8c71-b0b8b818cd11 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.703011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 376/760] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:49:59.704417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27137-0703-4624-b606-bc4b35ad7296 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.704708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27137-0703-4624-b606-bc4b35ad7296 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.704980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27137-0703-4624-b606-bc4b35ad7296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.734868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e9f27137-0703-4624-b606-bc4b35ad7296 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:49:59.736261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9f27137-0703-4624-b606-bc4b35ad7296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:49:59.740836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:59.748841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:49:59.749750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:49:59.750731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:49:59.751215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:49:59.817506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 377/761] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:49:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:49:59.818838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f27137-0703-4624-b606-bc4b35ad7296 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:49:59.827697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:49:59.829297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:49:59.907097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:49:59.960218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f27137-0703-4624-b606-bc4b35ad7296 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:49:59.972694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 385/762] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:49:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:49:59.979469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.036725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.036725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.036725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.036725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/786e4e7b1c5147e093693767b229e52b/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.037149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=786e4e7b1c5147e093693767b229e52b, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.037149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.059392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 378/763] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:49:59 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/786e4e7b1c5147e093693767b229e52b/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:50:00.060067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:00.142337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39018738-89dc-4c70-9a21-bb7e0f13aa74 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:00.177952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.184317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.185018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.185302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.185573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.199502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.201230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:00.231659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:00.232702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:00.233585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:00.233585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:00.233585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:00.233585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 386/764] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:00.240076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.293297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:00.351874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12247147-1500-419a-a0e2-c5916aba9b66 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:00.388621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874642-1144-45e7-88f1-6d8b6f3c86f1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:00.391229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.392249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.392788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.393262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.394061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.395685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:00.406493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:00.407594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:50:00.408460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:00.411328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 379/765] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:00.417657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.428648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.428920 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.429202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.430016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.430654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:00.430957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.432341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.446545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 387/766] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:00.452393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.524755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:00.575859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:00.591422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6796f4-8bfa-4cff-95bc-e77afa9d2349 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:00.595692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d8385-dd8e-428f-80b0-09eb76060e37 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:00.626989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.627671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.627950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.628408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.635607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.637170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:00.649194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:00.650237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:00.651279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 388/767] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:00.657394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.659216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.668035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.668403 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.668748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.669462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:00.669833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.671925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.677604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 380/768] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:00.688473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.803432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97ae62-a3d9-4a3c-b902-de024a022a2b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:00.834161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf310-2e85-48f7-b694-b6489240fea7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:00.874138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:00.885486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.886445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.886890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.887309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.888234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:00.888688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.890343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.893299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:00.893934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:00.894217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:00.894487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:00.895185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:00.895498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:00.896957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:00.897669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 381/769] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:00.903528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.925456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 389/770] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:00.927971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:00.935713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:01.067908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3369f4d-46fd-428e-b4aa-040861a3e203 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:01.075859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0184b-f142-41a1-8171-23f2d6824f96 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:01.114357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6d4e2-ccc5-4130-bbbb-3f45a05539b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:01.114981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6d4e2-ccc5-4130-bbbb-3f45a05539b5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.115360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6d4e2-ccc5-4130-bbbb-3f45a05539b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.115672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6d4e2-ccc5-4130-bbbb-3f45a05539b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.116238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5207091-67a4-4ad6-934e-46087b6cc767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:01.116388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6d4e2-ccc5-4130-bbbb-3f45a05539b5 None admin] RBAC: Adding query filter params (interface=public, service_id=04815675685c455da6de32fe06212ee9, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:01.116673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6d4e2-ccc5-4130-bbbb-3f45a05539b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:01.117806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207091-67a4-4ad6-934e-46087b6cc767 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.118269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207091-67a4-4ad6-934e-46087b6cc767 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.118593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6d4e2-ccc5-4130-bbbb-3f45a05539b5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:01.118838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207091-67a4-4ad6-934e-46087b6cc767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.119720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5207091-67a4-4ad6-934e-46087b6cc767 None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=452fcf1ae7a34bb1a2c1010575de2baa) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:01.120171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5207091-67a4-4ad6-934e-46087b6cc767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:01.121710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5207091-67a4-4ad6-934e-46087b6cc767 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:01.134775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 382/772] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/endpoints?service_id=04815675685c455da6de32fe06212ee9&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:01.134775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861fb1c-c0aa-4e2f-a574-675209578271 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.134775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861fb1c-c0aa-4e2f-a574-675209578271 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.134775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861fb1c-c0aa-4e2f-a574-675209578271 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.134775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 383/773] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:01.135572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 390/773] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:00 2022] GET /identity/v3/role_assignments?user.id=452fcf1ae7a34bb1a2c1010575de2baa&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:01.138160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861fb1c-c0aa-4e2f-a574-675209578271 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:01.146408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a8090-7919-4e88-a129-430e13bf8e36 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.146955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a8090-7919-4e88-a129-430e13bf8e36 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.147372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a8090-7919-4e88-a129-430e13bf8e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.186879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7b5a8090-7919-4e88-a129-430e13bf8e36 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:01.189283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7b5a8090-7919-4e88-a129-430e13bf8e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:01.271423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:01.289565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:01.297845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 391/774] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:01.368278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861fb1c-c0aa-4e2f-a574-675209578271 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:01.412254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc798f0d-43f5-4888-b218-ba6389217c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:01.413161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798f0d-43f5-4888-b218-ba6389217c5b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.413603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798f0d-43f5-4888-b218-ba6389217c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.414098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798f0d-43f5-4888-b218-ba6389217c5b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.414850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc798f0d-43f5-4888-b218-ba6389217c5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:01.416432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc798f0d-43f5-4888-b218-ba6389217c5b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:01.423665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 384/775] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:01 2022] GET /identity/v3/services => generated 772 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:01.603679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ded62-a40a-4e26-868c-fb71a73caf35 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.604206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ded62-a40a-4e26-868c-fb71a73caf35 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.604645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ded62-a40a-4e26-868c-fb71a73caf35 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.606164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 392/776] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:01.611943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bb929-1c1a-4433-ae31-e47d72cd497b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.612536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bb929-1c1a-4433-ae31-e47d72cd497b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.613009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bb929-1c1a-4433-ae31-e47d72cd497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.633055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5a4bb929-1c1a-4433-ae31-e47d72cd497b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:01.634562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5a4bb929-1c1a-4433-ae31-e47d72cd497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:01.675924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ded62-a40a-4e26-868c-fb71a73caf35 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:01.716135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 385/777] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:01.809238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ded62-a40a-4e26-868c-fb71a73caf35 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:01.867006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:01.867679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:01.867961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:01.868236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:01.884808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:01.889573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:01.890503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:01.891388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 393/778] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:01.896913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bb929-1c1a-4433-ae31-e47d72cd497b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:01.972446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:02.036070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bb929-1c1a-4433-ae31-e47d72cd497b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:02.087175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:02.087871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.088102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.091510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.091510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:02.095923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:02.097373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:02.108115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 386/779] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:02.113371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:02.186814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:02.201230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:02.274734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb894806-0bea-4f99-9a73-beb30f55147c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:02.307702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce294c1-075b-4284-a555-a16feb40f355 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:02.322143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:02.348193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:02.348900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.349178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.349448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] PATH_INFO: `/v3/services/network` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.350030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:02.351949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:02.361664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:02.362628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.363118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:02.363226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:02.364188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:02.365021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 387/780] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:02.366627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.370020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:02.387438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:02.389820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:02.417934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:02.418954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:02.419882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:02.419882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:02.419882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:02.422399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 394/781] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:02.428389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:02.485462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18e969-d337-4adb-ae5b-1801fb3ac55c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:02.523731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:02.524659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.524941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.525347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.525980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] RBAC: Adding query filter params (name=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:02.526281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:02.527993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:02.534209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 388/782] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:02.544064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:02.594151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:02.661436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea46824-f856-42f4-b8c1-6a1585e5979e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:02.702247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:02.702875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.703151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.703420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.704090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] RBAC: Adding query filter params (type=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:02.704390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:02.705863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:02.713376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 389/783] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:02.719081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:02.728385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba46873-daa4-4464-9aae-9ad8a23c337a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:02.748016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:02.808800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff300d0-8b9d-4eae-950e-f5443b555d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:02.809422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff300d0-8b9d-4eae-950e-f5443b555d80 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.809695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff300d0-8b9d-4eae-950e-f5443b555d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.809963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff300d0-8b9d-4eae-950e-f5443b555d80 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.810607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff300d0-8b9d-4eae-950e-f5443b555d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:02.810903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff300d0-8b9d-4eae-950e-f5443b555d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:02.812291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff300d0-8b9d-4eae-950e-f5443b555d80 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:02.838055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 395/784] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:02.839318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff123d-9c75-4871-abe1-9c6a94fc50ef None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.839654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff123d-9c75-4871-abe1-9c6a94fc50ef None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.839931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff123d-9c75-4871-abe1-9c6a94fc50ef None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.841875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 396/785] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:02.846434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f55e8-a04c-44cd-ac79-71a2e4e81dfa None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.846723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f55e8-a04c-44cd-ac79-71a2e4e81dfa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.847000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f55e8-a04c-44cd-ac79-71a2e4e81dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.865815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c99f55e8-a04c-44cd-ac79-71a2e4e81dfa None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:02.867680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c99f55e8-a04c-44cd-ac79-71a2e4e81dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:02.879265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfe528-7cac-430e-bd3b-30d0f484a224 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:02.882015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:02.927030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:02.927920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:02.928203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:02.928473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:02.929194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] RBAC: Adding query filter params (interface=public, service_id=4d179fc28e46498f95aabab8a116334f, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:02.929618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:02.931150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:02.937250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 390/786] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/endpoints?service_id=4d179fc28e46498f95aabab8a116334f&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:02.938106 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:02.947663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 397/787] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:02.948584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f55e8-a04c-44cd-ac79-71a2e4e81dfa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:03.055828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd65a36-7566-4000-868e-d4e94963ae6b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:03.091339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:03.091992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.095511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.095511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.100248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:03.107521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:03.109405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:03.110538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f55e8-a04c-44cd-ac79-71a2e4e81dfa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:03.127939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:03.128914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:03.129856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 391/788] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:03.135175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:03.154952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:03.155603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.155890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.156168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.156780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:03.158266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:03.164577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 398/789] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:02 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:03.199905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:03.305719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3b77b-8384-4e9f-a4f3-a993520a2c29 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:03.350645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:03.351292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.351598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.351871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.352607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:03.352907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:03.354338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:03.373131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 392/790] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 22:50:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:03.378833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:03.463806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:03.513830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c378c7e-b7dc-4e3a-b2f1-d45a360c8ba2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:03.561208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:03.576868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:03.577512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.577788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.578054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.592411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cdc3-03ec-42e6-bcff-575b963575c5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:03.605800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:03.607257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:03.646866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:03.648023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:03.649542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:03.649542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:03.649542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:03.657985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 399/791] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:03.663951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:03.676770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:03.677531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.677828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.678097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.692292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:03.703169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:03.708624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:03.709561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:03.710416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:03.713536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 393/792] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:03.720865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:03.813972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109883-a677-4bac-9e3e-39c28547d045 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:03.839813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21149-4542-44ab-bafc-e41d422dbe0f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:03.861648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295af268-48ae-449d-a119-d0cef3f61351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:03.862570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295af268-48ae-449d-a119-d0cef3f61351 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.863027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295af268-48ae-449d-a119-d0cef3f61351 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.864073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295af268-48ae-449d-a119-d0cef3f61351 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.865032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295af268-48ae-449d-a119-d0cef3f61351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:03.865637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295af268-48ae-449d-a119-d0cef3f61351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:03.868027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295af268-48ae-449d-a119-d0cef3f61351 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:03.879143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b83df-16ec-40a6-8577-1c54522589b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:03.879931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b83df-16ec-40a6-8577-1c54522589b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.880284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b83df-16ec-40a6-8577-1c54522589b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.880653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b83df-16ec-40a6-8577-1c54522589b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.881847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b83df-16ec-40a6-8577-1c54522589b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:03.881847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b83df-16ec-40a6-8577-1c54522589b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:03.883993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b83df-16ec-40a6-8577-1c54522589b2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:03.890602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 400/793] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:03.890602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344253-1798-4f23-a325-36293b672386 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.890602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344253-1798-4f23-a325-36293b672386 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.890602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344253-1798-4f23-a325-36293b672386 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.896302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 401/794] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:03.897345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7344253-1798-4f23-a325-36293b672386 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:03.903139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 394/795] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:50:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:03.904436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0e805-81fe-40dc-91fe-3753422d3e02 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:03.904726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0e805-81fe-40dc-91fe-3753422d3e02 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:03.905000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0e805-81fe-40dc-91fe-3753422d3e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:03.994020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.022005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7344253-1798-4f23-a325-36293b672386 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.057078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d4a0e805-81fe-40dc-91fe-3753422d3e02 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:04.058554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d4a0e805-81fe-40dc-91fe-3753422d3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:04.061408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.064395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.065678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.067872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.068242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.077567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.079471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:04.091730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:04.092667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:04.093673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 402/796] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:04.094158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.159598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 395/797] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:04.159598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b037bb-df73-450d-8147-0bed0c9fa722 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.159598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b037bb-df73-450d-8147-0bed0c9fa722 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.159598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b037bb-df73-450d-8147-0bed0c9fa722 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.159598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 396/798] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:04.159598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b037bb-df73-450d-8147-0bed0c9fa722 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.217882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba89a3d-63e6-4338-bbf4-ddb88d9ff190 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.255238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d9889-dc7a-439f-ae74-1fc76a15c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.256229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9889-dc7a-439f-ae74-1fc76a15c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.256581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9889-dc7a-439f-ae74-1fc76a15c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.256933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9889-dc7a-439f-ae74-1fc76a15c6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.268985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d9889-dc7a-439f-ae74-1fc76a15c6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.269604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.272429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d9889-dc7a-439f-ae74-1fc76a15c6f8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.281050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b037bb-df73-450d-8147-0bed0c9fa722 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-706d9889-dc7a-439f-ae74-1fc76a15c6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:04.292487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:04.293493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:04.294370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:04.294370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:04.294370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:04.294370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 403/799] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:04.296184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586c817-52a9-4927-ac02-c1d1780700c6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.296576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586c817-52a9-4927-ac02-c1d1780700c6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.296956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586c817-52a9-4927-ac02-c1d1780700c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.317977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2586c817-52a9-4927-ac02-c1d1780700c6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:04.319341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2586c817-52a9-4927-ac02-c1d1780700c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:04.327871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.328480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.328756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.329025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.329711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:04.330474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.331932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.340808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.348962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 397/800] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:04.349844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.409411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 404/801] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:04.411063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586c817-52a9-4927-ac02-c1d1780700c6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.504771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59528f23-6dba-4b68-a83a-31750e2a1de3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.547691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586c817-52a9-4927-ac02-c1d1780700c6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.549210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.550072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.550350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.550621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.552150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:04.552459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.553820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.568493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 398/802] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:04.569306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.571581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.592420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.593039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.593372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.593643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/e7ceb38886d34f9d8e59b2871989446c/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.595761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=e7ceb38886d34f9d8e59b2871989446c, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.597180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.612777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 405/803] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:50:04 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/e7ceb38886d34f9d8e59b2871989446c/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:50:04.613626 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.644305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.734879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640e4b8-30d5-4556-85a3-2ce5000e45d5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.740917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bfa20-8c49-4010-96f8-c67bac1e334a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.782461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.783532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.784037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.784505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.786585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.787354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.787672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.787944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.794750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.797003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.800467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:04.806028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:04.806975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:50:04.808218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:04.812402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 399/804] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:04.813438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:04.814342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:04.815253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:04.816229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:04.818876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 406/805] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:04.820150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.865858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.927328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac2ce3-6ae6-4dea-b54b-1d9426e5cc21 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.939874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b0ab5-79b9-41c5-a26d-076af5829fa7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:04.946721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:04.964906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.965765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.966315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.966740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.967642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.969172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.977928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:04.978649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:04.979025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:04.979182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:04.980359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 22:50:04.981272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:04.982063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:04.982063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:04.982063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:04.983335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:04.984244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 400/806] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:04.989917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:04.993058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 407/807] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:04.995542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.105357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf7ba9-6670-46de-83d5-f871e628ed51 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.122233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:05.126415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd00bc-7e7f-410f-9ed9-6a61065604a8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.149232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.150401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.150956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.151597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.152651 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:05.153754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.156456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.168916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.169815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.170565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.171058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.171924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:05.172446 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.174003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.187760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 401/809] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:05.187760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.188614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 408/809] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 22:50:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:05.189853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.228893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:05.292846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e026-222d-4fe5-8a81-dd50432ff240 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.329335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.329572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f443b-3f86-4dc2-80a2-f3c0231b42c1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.330970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.331563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.332788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.345359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.347776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:05.367039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:05.368216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:05.369104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:05.369104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:05.369104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:05.372833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.373654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.373957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 402/810] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:05.374294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.374746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.374974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.417612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:05.473244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.475348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:05.477084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.493220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b97ae1-528a-4f2f-ac88-32fddd161b7b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.538033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.538792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.539350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.539526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.540271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:05.540682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.542183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.552010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 403/811] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 22:50:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:05.552905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.606754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 409/812] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:50:05.607916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.671336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dc801-d894-4635-94a3-96d8e88045a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.678696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:05.707987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.708603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.708887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.709162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.709803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:05.710107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.711589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.723817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 404/813] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:05.724634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.734311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a43f1-cc68-4a8c-8603-a5078487c518 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.748730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:05.777713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b282b-5a6e-4c80-90c3-07a90e75eb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.778699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b282b-5a6e-4c80-90c3-07a90e75eb8c None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.779219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b282b-5a6e-4c80-90c3-07a90e75eb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.779417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b282b-5a6e-4c80-90c3-07a90e75eb8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.780123 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b282b-5a6e-4c80-90c3-07a90e75eb8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.781730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b282b-5a6e-4c80-90c3-07a90e75eb8c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.806472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 410/814] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:05.808673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078512-72a0-4a82-821a-8ac4434cd84d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.809019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078512-72a0-4a82-821a-8ac4434cd84d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.809282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078512-72a0-4a82-821a-8ac4434cd84d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.814791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 411/815] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:05.814791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078512-72a0-4a82-821a-8ac4434cd84d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:05.849556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ca37f-11de-4481-bfae-5bbac9fe36ec None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:05.902275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a7681-7e62-422e-b93b-46259b3f6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:05.903360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a7681-7e62-422e-b93b-46259b3f6350 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.903948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a7681-7e62-422e-b93b-46259b3f6350 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.904482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a7681-7e62-422e-b93b-46259b3f6350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.917502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a7681-7e62-422e-b93b-46259b3f6350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:05.919516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a7681-7e62-422e-b93b-46259b3f6350 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4f8a7681-7e62-422e-b93b-46259b3f6350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:05.940827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:05.941866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:05.942774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:05.942774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:05.942774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:05.942774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 405/816] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:05.943672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96006b20-eb58-4018-8e48-bd1a0f58b358 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:05.943955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96006b20-eb58-4018-8e48-bd1a0f58b358 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:05.944223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96006b20-eb58-4018-8e48-bd1a0f58b358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:05.965618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-96006b20-eb58-4018-8e48-bd1a0f58b358 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:05.966838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-96006b20-eb58-4018-8e48-bd1a0f58b358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:06.046321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:06.058532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 406/817] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:06.059435 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96006b20-eb58-4018-8e48-bd1a0f58b358 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.086147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078512-72a0-4a82-821a-8ac4434cd84d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.111881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:06.134355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.136227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.136585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.136945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.145079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.146605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:06.163006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:06.164537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:06.165969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 412/818] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:50:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:06.170300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.201673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96006b20-eb58-4018-8e48-bd1a0f58b358 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.265561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.266178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.266459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.266732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.267379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:06.267709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.269079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.291150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 407/819] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:06.298112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.363700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:06.369715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583748e5-4563-4dd4-b52e-ccd423e10e24 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.412755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117a4f6-d094-4ea4-bb7d-e274244a182f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.417651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.418263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.418545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.418816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.419567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:06.419875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.421556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.444348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 413/820] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:06.445319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.462174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.463295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.463873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.464159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.473622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.475506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.489594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:06.497057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:06.498163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:06.499279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:06.505551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 408/821] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:06.505551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.571767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610564a1-ab51-4b1f-9c1f-36d86c0fccf1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.610532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.611944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.612537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.613131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] PATH_INFO: `/v3/services/network` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.613972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.615831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.619818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c437b-c9c8-40f8-803a-6dd08eeb6ad1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:06.626574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:06.627611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 22:50:06.628482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:06.635959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 414/822] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:06.636672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.638242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:06.665374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.666017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.666299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.666729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.681302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.682768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:06.700901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:06.701925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:06.702829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:06.702829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:06.702829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:06.702829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 409/823] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:06.703535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.760757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de898260-e189-478e-a5ca-a12779fc1a7a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.762522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:06.808106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.809008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.809442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.809860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.810691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:06.811143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.812824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.831243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 415/824] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:06.832646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.840833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb183d-b4cc-49c3-a2fe-0a669647ac5a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:06.883353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:06.884343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:06.884846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:06.885329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:06.886158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] RBAC: Adding query filter params (name=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:06.886697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:06.888419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:06.898525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 410/825] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:06.899689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:06.932298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:06.966298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3637b8-3831-430b-99b5-1348f7f3d92f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.001902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.002510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.002792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.003528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.004163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:07.004468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:07.005823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:07.015436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5b4fd-5710-44e0-85e4-ce03bc390328 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.017816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 416/826] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:07.018693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:07.042387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:07.053404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccb778-6fe5-4878-a050-0bdf7f332af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.054542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccb778-6fe5-4878-a050-0bdf7f332af3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.054823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccb778-6fe5-4878-a050-0bdf7f332af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.055099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccb778-6fe5-4878-a050-0bdf7f332af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.055852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccb778-6fe5-4878-a050-0bdf7f332af3 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=632d504031cb466299331db2d9184912, user.id=786e4e7b1c5147e093693767b229e52b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:07.056162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccb778-6fe5-4878-a050-0bdf7f332af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:07.057615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccb778-6fe5-4878-a050-0bdf7f332af3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:07.063730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 411/827] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:06 2022] GET /identity/v3/role_assignments?user.id=786e4e7b1c5147e093693767b229e52b&scope.project.id=632d504031cb466299331db2d9184912&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:07.064895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19274ad-9314-43a6-bc35-87fd71d3c3d7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.065208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19274ad-9314-43a6-bc35-87fd71d3c3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.065480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19274ad-9314-43a6-bc35-87fd71d3c3d7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.066749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 412/828] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:07.067483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19274ad-9314-43a6-bc35-87fd71d3c3d7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:07.155773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938a23f-685c-42f4-a585-665a3a0efb96 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.194512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:07.204904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19274ad-9314-43a6-bc35-87fd71d3c3d7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.208792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03074604-633d-473b-9600-e5260403d9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.209627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03074604-633d-473b-9600-e5260403d9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.210074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03074604-633d-473b-9600-e5260403d9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.211044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03074604-633d-473b-9600-e5260403d9b4 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.211906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03074604-633d-473b-9600-e5260403d9b4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:07.212379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03074604-633d-473b-9600-e5260403d9b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:07.213936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03074604-633d-473b-9600-e5260403d9b4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:07.221891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 417/829] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:07.223402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab09c04-3689-4154-9fed-99d66081a1e7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.223897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab09c04-3689-4154-9fed-99d66081a1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.224306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab09c04-3689-4154-9fed-99d66081a1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.248658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6ab09c04-3689-4154-9fed-99d66081a1e7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:07.250191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6ab09c04-3689-4154-9fed-99d66081a1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:07.252745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.253601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.254671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.254671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.264076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:07.265591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:07.277953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:07.279061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:07.280225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:07.283594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 413/830] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:07.283594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:07.343466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 418/831] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:07.344401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab09c04-3689-4154-9fed-99d66081a1e7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:07.362562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:07.428012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfb99c-9630-46a3-830d-9ba38a6d3d19 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.490836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.492565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.493200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.494228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.495075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:07.496811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:07.525579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab09c04-3689-4154-9fed-99d66081a1e7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.543738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 414/832] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:07 2022] POST /identity/v3/endpoints => generated 337 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:07.585671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.586609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.587045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.587543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.588412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:07.588874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:07.590534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:07.605297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 419/833] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:07.611815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:07.628194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:07.751750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa560-c055-4061-b051-6becdf97c236 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.791964 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.792712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.792998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.793274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/452fcf1ae7a34bb1a2c1010575de2baa/roles/8d7867fc676d4963a98f9eaed5f9f9f6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:07.801571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:07.823402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=452fcf1ae7a34bb1a2c1010575de2baa, role_id=8d7867fc676d4963a98f9eaed5f9f9f6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:07.825393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:07.851908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 415/834] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:50:07 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/452fcf1ae7a34bb1a2c1010575de2baa/roles/8d7867fc676d4963a98f9eaed5f9f9f6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:50:07.918372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:07.953394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934e63-d38f-4820-8469-862623fac2b2 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:07.995743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:07.996318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:07.996602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:07.996871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.012436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:08.019130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:08.020142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:08.020993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:08.024475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 420/835] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:08.030583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:08.113943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:08.171608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c823b-12a7-4d76-bb37-da0fc3c51797 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:08.208024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556ad70-dd17-4782-bbe2-1de2efbd231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:08.208660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556ad70-dd17-4782-bbe2-1de2efbd231f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.208898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556ad70-dd17-4782-bbe2-1de2efbd231f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.209170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556ad70-dd17-4782-bbe2-1de2efbd231f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.209839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556ad70-dd17-4782-bbe2-1de2efbd231f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:08.210138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556ad70-dd17-4782-bbe2-1de2efbd231f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:08.212102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556ad70-dd17-4782-bbe2-1de2efbd231f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:08.220740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 416/836] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:08.226175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:08.269566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6978b4-831b-4833-8d38-b9c35baa20eb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.269912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6978b4-831b-4833-8d38-b9c35baa20eb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.270130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6978b4-831b-4833-8d38-b9c35baa20eb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.271612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 417/837] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:08.277025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6774f6-3c02-4b02-8ee4-8eb5c12c241d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.277314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6774f6-3c02-4b02-8ee4-8eb5c12c241d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.277587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6774f6-3c02-4b02-8ee4-8eb5c12c241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.298515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6d6774f6-3c02-4b02-8ee4-8eb5c12c241d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:08.300033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d6774f6-3c02-4b02-8ee4-8eb5c12c241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:08.336210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5032bee-579d-4b72-9dcb-aebfeb1d6758 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:08.368630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:08.375538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:08.376183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.376603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.376889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.388359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 418/838] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:08.389691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:08.391233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:08.410101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:08.411070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:08.412003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:08.412003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:08.412003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:08.412003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 421/839] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:08.417673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6774f6-3c02-4b02-8ee4-8eb5c12c241d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:08.473548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:08.525587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6774f6-3c02-4b02-8ee4-8eb5c12c241d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:08.561281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:08.561946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.562226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.562504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.563178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:08.563483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:08.564927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:08.577824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 419/840] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:08.584794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:08.658127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:08.693826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6f00-dabc-4600-bdef-c4b0a9e25ced None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:08.733038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:08.733909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.734279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.734643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.743451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:08.745136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:08.758085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:08.760148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 22:50:08.761313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:08.764183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 422/841] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:08.765371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:08.775950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d20c9-16a8-4a39-a0b5-b0016c0d298c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:08.790232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:08.817937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:08.818861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.819145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.819425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.820976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:08.821399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:08.823214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:08.835852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:08.836860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:08.837700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:08.838633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 420/842] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:08.844587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:08.901469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705877d-9828-4e05-9396-83e18aafa0d4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:08.947926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:08.948587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:08.948873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:08.949139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:08.949895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:08.950201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:08.951795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:08.963480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c50398-ad8b-4338-97c7-c9c821578bfa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:08.976490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 423/843] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:08.979455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:09.005473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:09.005974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.006251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.006520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.007689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:09.007991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:09.009511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:09.018775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 421/844] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:09.020054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:09.064090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:09.109405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97547f91-f7ab-45de-867f-e326847b5d8a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:09.111143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:09.146991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ee831-1788-42d6-9013-82a9d656f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:09.147631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ee831-1788-42d6-9013-82a9d656f560 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.147969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ee831-1788-42d6-9013-82a9d656f560 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.148236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ee831-1788-42d6-9013-82a9d656f560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.148956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c3ef2-d340-4935-88b7-78b8c3d40fef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:09.196310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:09.197777 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.198529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.199027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.200127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:09.200435 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:09.202658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:09.215715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 422/845] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:09.226737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ee831-1788-42d6-9013-82a9d656f560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:09.228719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ee831-1788-42d6-9013-82a9d656f560 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:09.348425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 424/846] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:50:09.354851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:09.465423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e564f-9227-4501-86fb-da8c30c8296e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.465771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e564f-9227-4501-86fb-da8c30c8296e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.465973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e564f-9227-4501-86fb-da8c30c8296e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.467422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 425/847] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:09.468212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cb994-e80c-43fb-a9f4-14744ec98a4e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:09.472815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d965c3-418b-4f9b-95c3-e50955e9c50b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.473351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d965c3-418b-4f9b-95c3-e50955e9c50b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.473841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d965c3-418b-4f9b-95c3-e50955e9c50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.476440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:09.481193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:09.498943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-43d965c3-418b-4f9b-95c3-e50955e9c50b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:09.500537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-43d965c3-418b-4f9b-95c3-e50955e9c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:09.512790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:09.513437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.513719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.513993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.529047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:09.531174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:09.550314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:09.551320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:09.552599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:09.552599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:09.552599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:09.552599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 423/848] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:09.564356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:09.581565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 426/849] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:09.670967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a8b03-577c-4830-9432-fd8afbe14751 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:09.705977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:09.706593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.706872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.707143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.707820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:09.708122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:09.709488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:09.722284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 424/850] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:09.728325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d965c3-418b-4f9b-95c3-e50955e9c50b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:09.837497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d965c3-418b-4f9b-95c3-e50955e9c50b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:09.868577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:09.874534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:09.875165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:09.875441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:09.875735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:09.880543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:09.883671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:09.885580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:09.898198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:09.899133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:09.900144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:09.901451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:09.903076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 427/851] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:09.911521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:09.990575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd461c-2791-4c5c-b55d-bba52c14eb37 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:10.016013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd0ee1-433a-4b0e-a86c-d0966919c8c0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:10.029823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:10.030470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.030753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.031022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.031635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:10.033009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:10.043353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:10.044325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:10.045465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:10.046202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 425/852] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 9 22:50:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:10.055033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:10.055686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.055963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.056228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.056911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:10.057207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:10.057566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:10.058617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:10.071426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 428/853] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:10.079678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:10.152506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:10.180433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:10.203609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f5068-42fd-4cab-8ff9-d027051496ea None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:10.222192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9cb60-4bcb-44f5-9ada-9409558bb293 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:10.239563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:10.240190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.240446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.240771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.241364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:10.242321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:10.244155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:10.250122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 426/854] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:10.255474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:10.268902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1f08-3ae5-477e-8825-776add73183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:10.269776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1f08-3ae5-477e-8825-776add73183e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.270626 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1f08-3ae5-477e-8825-776add73183e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.271068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1f08-3ae5-477e-8825-776add73183e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.272321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea1f08-3ae5-477e-8825-776add73183e None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=632d504031cb466299331db2d9184912, user.id=e7ceb38886d34f9d8e59b2871989446c) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:10.273324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea1f08-3ae5-477e-8825-776add73183e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:10.276133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea1f08-3ae5-477e-8825-776add73183e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:10.291806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 429/855] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:10 2022] GET /identity/v3/role_assignments?user.id=e7ceb38886d34f9d8e59b2871989446c&scope.project.id=632d504031cb466299331db2d9184912&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:10.297434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa93d4-1c02-421d-a92c-f6b709772e21 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.297732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa93d4-1c02-421d-a92c-f6b709772e21 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.297979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa93d4-1c02-421d-a92c-f6b709772e21 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.299529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 430/856] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:10.305850 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69ea6a-3cc2-4f4d-a339-77be401b6e42 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.306159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69ea6a-3cc2-4f4d-a339-77be401b6e42 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.306390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69ea6a-3cc2-4f4d-a339-77be401b6e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.325680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0f69ea6a-3cc2-4f4d-a339-77be401b6e42 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:10.327018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0f69ea6a-3cc2-4f4d-a339-77be401b6e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:10.383754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d04db-b1b8-4d9b-ba8b-d536b0506f00 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:10.416009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 431/857] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:10.429344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:10.429975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.430263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.430538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.431181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:10.431505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:10.432913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:10.438559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 427/858] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:10.689827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69ea6a-3cc2-4f4d-a339-77be401b6e42 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:10.742103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:10.798631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69ea6a-3cc2-4f4d-a339-77be401b6e42 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:10.834106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:10.834746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:10.835026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:10.835859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:10.846328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:10.851338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:10.852326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:10.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 432/859] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:50:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:10.858589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:10.871428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:10.976911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fde048-f285-4bba-adcb-9313e94bc7b0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:11.012298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:11.012920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.013199 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.013471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.014135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:11.014436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:11.015859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:11.024576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 428/860] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:50:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:11.029919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:11.140397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eead167-bbb7-42db-9317-5e9446592356 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:11.148749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:11.178830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:11.179516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.179820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.180089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.191322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:11.193872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:11.215807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:11.216791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:11.218096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:11.218096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:11.218096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:11.218096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 433/861] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:11.226427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:11.301772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:11.338094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818fc6f-ff85-4ecd-9bf6-77e769b8169d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:11.376151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86501015-3102-4701-a59e-e1b8158cb586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:11.376771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501015-3102-4701-a59e-e1b8158cb586 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.377053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501015-3102-4701-a59e-e1b8158cb586 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.377326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501015-3102-4701-a59e-e1b8158cb586 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.377970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86501015-3102-4701-a59e-e1b8158cb586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:11.378268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86501015-3102-4701-a59e-e1b8158cb586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:11.379672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86501015-3102-4701-a59e-e1b8158cb586 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:11.394013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 429/862] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:11.399623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:11.513899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c085a-be00-4e1f-bff2-05fff472b6ce None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:11.538137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e05e8c-42a4-428f-a397-f3219e13895f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.538727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e05e8c-42a4-428f-a397-f3219e13895f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.539041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e05e8c-42a4-428f-a397-f3219e13895f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.541451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 430/863] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:11.547130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cecea6-7404-4dec-a794-67fad9e07fd8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.547425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cecea6-7404-4dec-a794-67fad9e07fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.547726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cecea6-7404-4dec-a794-67fad9e07fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.552078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:11.552651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.552934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.553212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.571848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b4cecea6-7404-4dec-a794-67fad9e07fd8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:11.572772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:11.573051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b4cecea6-7404-4dec-a794-67fad9e07fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:11.574694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:11.594207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:11.595223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:11.596212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:11.597711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 434/864] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:50:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:11.603313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:11.626735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:11.671945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 431/865] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:11.719245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda314b-4384-4ffb-95d1-988247008456 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:11.755154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:11.755847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.756126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.756399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.757177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:11.757485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:11.758900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:11.778205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 435/866] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 22:50:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:11.783647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cecea6-7404-4dec-a794-67fad9e07fd8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:11.844613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:11.886675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cecea6-7404-4dec-a794-67fad9e07fd8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:11.928505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:11.929159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:11.929436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:11.929705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:11.949946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:12.003520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:12.005009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:12.047994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:12.074497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:12.078699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d37c0-7054-46b4-bd8d-ebf5aec59379 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:12.121857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:12.122819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.123251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.123701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.124502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:12.126013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:12.126331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 432/867] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 22:50:12.131467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:12.155783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 436/868] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:50:11 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:50:12.257857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfee87f-a963-4ca5-8cd3-2cc5e66be899 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:12.308940 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:12.309627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.309866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.310145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.322262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:12.323405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:12.339281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:12.340325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:12.341233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:12.341233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:12.341233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:12.341233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 433/869] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:12.347193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:12.455088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaee05-39f7-4911-9047-f783159070e3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:12.491157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7e7be-2468-47a4-9488-e47714acada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:12.491831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e7be-2468-47a4-9488-e47714acada5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.492107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e7be-2468-47a4-9488-e47714acada5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.492381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e7be-2468-47a4-9488-e47714acada5 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.493089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7e7be-2468-47a4-9488-e47714acada5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:12.493323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7e7be-2468-47a4-9488-e47714acada5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:12.494690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7e7be-2468-47a4-9488-e47714acada5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:12.497179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:12.509653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 437/870] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:12.515267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:12.527273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:12.628329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f7df9-1580-4fe1-b17e-e651e67f565a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:12.667102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663ab42-e016-49ee-9d6d-899321479763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:12.667758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663ab42-e016-49ee-9d6d-899321479763 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.668035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663ab42-e016-49ee-9d6d-899321479763 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.668302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663ab42-e016-49ee-9d6d-899321479763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.675477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663ab42-e016-49ee-9d6d-899321479763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:12.677278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663ab42-e016-49ee-9d6d-899321479763 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:12.684847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ae9e-c51b-4bae-9762-6c129b0a384b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.685405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ae9e-c51b-4bae-9762-6c129b0a384b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.685928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80ae9e-c51b-4bae-9762-6c129b0a384b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0663ab42-e016-49ee-9d6d-899321479763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:12.690803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:12.691749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:12.692660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 434/871] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:12.693744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 438/872] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:12.699371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39783b-7b29-4df9-9736-34d7964f943a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.699686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39783b-7b29-4df9-9736-34d7964f943a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.699960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39783b-7b29-4df9-9736-34d7964f943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.700166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ae9e-c51b-4bae-9762-6c129b0a384b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:12.718969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5d39783b-7b29-4df9-9736-34d7964f943a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:12.720164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5d39783b-7b29-4df9-9736-34d7964f943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:12.786387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:12.795623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 435/873] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:12.797711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feaede-0faf-4a2d-84a5-9b41ff7d1ef2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.798594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feaede-0faf-4a2d-84a5-9b41ff7d1ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.799010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feaede-0faf-4a2d-84a5-9b41ff7d1ef2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.805192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 436/874] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:12.806482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6262df-df17-445e-9cc0-c00790ad881a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.806765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6262df-df17-445e-9cc0-c00790ad881a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.807040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6262df-df17-445e-9cc0-c00790ad881a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.828323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7f6262df-df17-445e-9cc0-c00790ad881a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:12.829810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f6262df-df17-445e-9cc0-c00790ad881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:12.830174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80ae9e-c51b-4bae-9762-6c129b0a384b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:12.870497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:12.872057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:12.872525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:12.872994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:12.873919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:12.874402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:12.877271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:12.892469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 439/875] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:12.897755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:12.917874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 437/876] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:12.961783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:13.040740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c73c5-596e-430b-a97c-75bd9065e1a8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:13.066595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6262df-df17-445e-9cc0-c00790ad881a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:13.098752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:13.099504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:13.099786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:13.100055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:13.100793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=452fcf1ae7a34bb1a2c1010575de2baa) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:13.101654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:13.103082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:13.109445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 440/877] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:12 2022] GET /identity/v3/role_assignments?user.id=452fcf1ae7a34bb1a2c1010575de2baa&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:13.182265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6262df-df17-445e-9cc0-c00790ad881a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:13.213567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:13.223311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:13.224273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:13.224762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:13.225218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:13.226020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:13.227582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:13.237611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:13.238857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:13.239849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:13.240922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 438/878] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:13.248467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:13.253541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:13.367347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bcf77-73d5-4730-b264-e4cebf9d82b6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:13.383436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fad3a4-70e7-4f9d-bc24-eb30a5e17fa1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:13.409694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:13.410506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969ea71-120f-49cd-adde-016854d5d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:13.411407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ea71-120f-49cd-adde-016854d5d1c8 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:13.411837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ea71-120f-49cd-adde-016854d5d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:13.412118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ea71-120f-49cd-adde-016854d5d1c8 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:13.413000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969ea71-120f-49cd-adde-016854d5d1c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:13.414907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969ea71-120f-49cd-adde-016854d5d1c8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:13.426806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:13.427443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:13.427769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:13.428104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:13.428843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:13.429132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:13.430543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:13.436106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 441/880] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:13.439578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 439/880] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:50:13 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:50:13.441973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:13.564482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c68be-e9bd-4fb9-a826-98bfe3e73120 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:13.609436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bacc9-0d9a-46d9-81d6-760b59d8ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:13.610459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bacc9-0d9a-46d9-81d6-760b59d8ed8a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:13.610942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bacc9-0d9a-46d9-81d6-760b59d8ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:13.611407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bacc9-0d9a-46d9-81d6-760b59d8ed8a None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:13.612418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bacc9-0d9a-46d9-81d6-760b59d8ed8a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:13.612938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bacc9-0d9a-46d9-81d6-760b59d8ed8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:13.614540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bacc9-0d9a-46d9-81d6-760b59d8ed8a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:13.620466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 442/881] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:14.489562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841a04-4765-49ad-aca9-8535051c6c87 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:14.490200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841a04-4765-49ad-aca9-8535051c6c87 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:14.490200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841a04-4765-49ad-aca9-8535051c6c87 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:14.491793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 440/882] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:14.497068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c578e6-d86c-4f61-8f01-464a34ae0613 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:14.497557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c578e6-d86c-4f61-8f01-464a34ae0613 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:14.497653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c578e6-d86c-4f61-8f01-464a34ae0613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:14.517381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c0c578e6-d86c-4f61-8f01-464a34ae0613 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:14.518575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c0c578e6-d86c-4f61-8f01-464a34ae0613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:14.558916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:14.585931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 443/883] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:14.848915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e841a04-4765-49ad-aca9-8535051c6c87 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:14.903931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:14.953767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e841a04-4765-49ad-aca9-8535051c6c87 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:14.987031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:14.987683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:14.987960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:14.988226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:14.988804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:14.990160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:14.998920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:14.999981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:15.000872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 441/884] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:15.006149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c578e6-d86c-4f61-8f01-464a34ae0613 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:15.111816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c578e6-d86c-4f61-8f01-464a34ae0613 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:15.147313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:15.148360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.148860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.149333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.150285 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:15.150807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:15.152393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:15.157565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 444/885] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:15.162878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:15.266745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989da69-24c3-4eb1-aa83-5969494a696d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:15.283866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:15.302617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:15.303225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.303514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.304708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.305354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:15.305654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:15.307018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:15.312569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 442/886] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:15.318322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:15.359317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:15.425651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395eab9-4d95-44b7-9d92-247133744ff9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:15.461399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858f7f4-45cf-4234-a53a-ee5fe81e00ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:15.462013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858f7f4-45cf-4234-a53a-ee5fe81e00ba None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.462294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858f7f4-45cf-4234-a53a-ee5fe81e00ba None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.462579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858f7f4-45cf-4234-a53a-ee5fe81e00ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.463261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858f7f4-45cf-4234-a53a-ee5fe81e00ba None admin] RBAC: Adding query filter params (interface=public, service_id=80fb073beb184f259a176629c9fe29f9, region_id=RegionOne) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:15.463598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858f7f4-45cf-4234-a53a-ee5fe81e00ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:15.464958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858f7f4-45cf-4234-a53a-ee5fe81e00ba None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:15.473043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 445/887] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:50:15 2022] GET /identity/v3/endpoints?service_id=80fb073beb184f259a176629c9fe29f9&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:15.478745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:15.519402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc28f5-ff4a-4636-aa4a-55cdc4b16205 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.519968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc28f5-ff4a-4636-aa4a-55cdc4b16205 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.519968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc28f5-ff4a-4636-aa4a-55cdc4b16205 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.521560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 446/888] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:15.527003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b376b-edb2-4ca9-8ebd-e0746bd2d484 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.527288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b376b-edb2-4ca9-8ebd-e0746bd2d484 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.527595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b376b-edb2-4ca9-8ebd-e0746bd2d484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.548048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8f9b376b-edb2-4ca9-8ebd-e0746bd2d484 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:15.549545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8f9b376b-edb2-4ca9-8ebd-e0746bd2d484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:15.589540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab129cc2-e903-497b-b032-c35965cee2d7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:15.627407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81874a4e-edf6-4e77-97bd-cf04ebfeb121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:15.628416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81874a4e-edf6-4e77-97bd-cf04ebfeb121 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.628901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81874a4e-edf6-4e77-97bd-cf04ebfeb121 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.629498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81874a4e-edf6-4e77-97bd-cf04ebfeb121 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.630288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81874a4e-edf6-4e77-97bd-cf04ebfeb121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:15.632023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81874a4e-edf6-4e77-97bd-cf04ebfeb121 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:15.635553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 447/889] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:15.644932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 443/890] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:15 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 22:50:15.786532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97824d-b103-4525-939c-fb65497b50f1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.786921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97824d-b103-4525-939c-fb65497b50f1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.787099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97824d-b103-4525-939c-fb65497b50f1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.789296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 448/891] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:15.794922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bfee6-7ccf-48fc-96b1-0a80422d024a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.795460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bfee6-7ccf-48fc-96b1-0a80422d024a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.795771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bfee6-7ccf-48fc-96b1-0a80422d024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.815263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-274bfee6-7ccf-48fc-96b1-0a80422d024a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:15.816520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-274bfee6-7ccf-48fc-96b1-0a80422d024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:15.861105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:15.871924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b997d0f-7fd8-4c39-a3ad-8d156a05f329 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.872275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b997d0f-7fd8-4c39-a3ad-8d156a05f329 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.872489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b997d0f-7fd8-4c39-a3ad-8d156a05f329 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.874060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 449/892] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:15.879008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b827-fbdb-4c94-828e-7d4fb0642df2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:15.879291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b827-fbdb-4c94-828e-7d4fb0642df2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:15.879609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b827-fbdb-4c94-828e-7d4fb0642df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:15.892319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 444/893] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:15.903852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0100b827-fbdb-4c94-828e-7d4fb0642df2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:15.905046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0100b827-fbdb-4c94-828e-7d4fb0642df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:15.917598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bfee6-7ccf-48fc-96b1-0a80422d024a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:15.933299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:15.985846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 450/894] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:16.032283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bfee6-7ccf-48fc-96b1-0a80422d024a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.071185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.072310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.072881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.073175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.085714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:16.090849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:16.091849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:16.092887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 445/895] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 22:50:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:16.098141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100b827-fbdb-4c94-828e-7d4fb0642df2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.182228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.205635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100b827-fbdb-4c94-828e-7d4fb0642df2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.225093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:16.249471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.250321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.250629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.250911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.251663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:16.252089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.253491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.263363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 451/896] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:16.267544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.296963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:16.304679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c71d1-e032-4954-afa8-30304ad75ef0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.343069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.343723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.344002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.344273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.344852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.346228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:16.355723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:16.356675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:16.357565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 446/897] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:16.359076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.381943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f116c2e-f296-4009-8e68-634ea09acb09 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.424043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.424790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.425060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.425448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.426105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.427473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.447528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 452/898] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:50:16 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:16.447528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.465879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:16.470699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd5a0d-339b-4079-8f4b-2c559b195db8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.525146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.526397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.527084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.527578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.543574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.546245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.556840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:16.570026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:16.571232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:16.572359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:16.572359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:16.572359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:16.574364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 447/899] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:16.580139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.588895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d45da-6925-401e-8d98-c2fd9e59e3ab None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.633415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.634185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.634569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.634961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.635745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:16.636149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.637740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.650175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 453/900] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:16.651875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.711900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa1353-4770-48aa-95c8-830300f421f1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.755360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.756009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.756287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.756554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.757731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:16.758035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.759446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.771968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 448/901] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:16.777709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.780703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fa7b4-bcb9-47ae-8482-ad9145a5caa9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.790125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:16.825856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.826470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.826752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.827027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.827682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:16.827989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.830325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.836190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 454/902] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:16.842101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:16.859117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:16.903006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67e61-8a75-46ce-bf60-e55028c2b600 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.941161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:16.941955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:16.942312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:16.942686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:16.950575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:16.952256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:16.958774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eecf0-60ed-42b1-aaa0-c2fe4999b673 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:16.966823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:16.967961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:16.968904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:16.970485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 449/903] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:16.976042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:17.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:17.006959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.008263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.009057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.010766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] RBAC: Adding query filter params (interface=public, service_id=2b6c0bbae6bb4340b62f444a9e497d35, region_id=RegionOne) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:17.011585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:17.014711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:17.031901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 455/904] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/endpoints?service_id=2b6c0bbae6bb4340b62f444a9e497d35&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:17.034182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:17.051613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:17.088587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a9db9-231f-4ed4-9daf-e6e90fad7988 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:17.123196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:17.124029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.124309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.124576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.125852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:17.125852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:17.127006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:17.144558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 450/905] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 22:50:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:17.148841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:17.150267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:17.160430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b171-b5cd-459a-b7ce-875618fdf47d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:17.198439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:17.199096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.199374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.199688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.200835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:17.201688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:17.215710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 456/906] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:17 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 22:50:17.277879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b88527-f072-4963-aa02-ab9a8aff171d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:17.306478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:17.320740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:17.321780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.322421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.322911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.401557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:17.403363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:17.515283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 451/907] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 22:50:17.520729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:17.626319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae190c3-bb31-4726-9822-c3a6cda716b0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:17.666234 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:17.667242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.667745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.668215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.677687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:17.680669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:17.682497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:17.702490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:17.703514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:17.704395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:17.704395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:17.704395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:17.706597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 457/908] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:17.721006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:17.753562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:17.839062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310297b-6ebb-4f43-ba3e-63487d17a2bc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:17.874901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb0b2b-aa89-4800-a750-3d7d8ca77153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:17.875531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb0b2b-aa89-4800-a750-3d7d8ca77153 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.875815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb0b2b-aa89-4800-a750-3d7d8ca77153 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.876084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb0b2b-aa89-4800-a750-3d7d8ca77153 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.876781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb0b2b-aa89-4800-a750-3d7d8ca77153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:17.877026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb0b2b-aa89-4800-a750-3d7d8ca77153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:17.878378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb0b2b-aa89-4800-a750-3d7d8ca77153 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:17.890576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 452/909] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:17.896244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:17.919835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c039be-3529-439f-a06a-5f692d51e8ca None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.920412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c039be-3529-439f-a06a-5f692d51e8ca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.920648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c039be-3529-439f-a06a-5f692d51e8ca None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.922229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 453/910] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:17.927312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00917d85-db5b-456d-b607-ad7ae7588aa7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:17.927630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00917d85-db5b-456d-b607-ad7ae7588aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:17.927907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00917d85-db5b-456d-b607-ad7ae7588aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:17.947543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-00917d85-db5b-456d-b607-ad7ae7588aa7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:17.951530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-00917d85-db5b-456d-b607-ad7ae7588aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:18.012596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b9cb7-5ac1-481b-ac79-4a3c675c2178 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:18.047531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 454/911] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:18.055319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:18.055977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.056256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.056521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.063334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:18.064790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:18.076013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:18.076894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:18.077770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 458/912] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:18.083445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00917d85-db5b-456d-b607-ad7ae7588aa7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:18.102417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:18.199448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00917d85-db5b-456d-b607-ad7ae7588aa7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:18.235277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:18.236274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.236771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.237240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.238119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:18.238624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:18.240301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:18.252516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 455/913] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:18.259517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:18.321122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:18.341139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:18.375822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff85ad-7ae9-4aca-83df-c4a99ddb2004 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:18.414444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:18.414965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.415315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.415631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.416362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] RBAC: Adding query filter params (role.id=eb6adc880d6b4bbcb153617ac3cd76ff, scope.project.id=632d504031cb466299331db2d9184912, user.id=452fcf1ae7a34bb1a2c1010575de2baa) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:18.416672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:18.418175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:18.421101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:18.426016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 459/914] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:18 2022] GET /identity/v3/role_assignments?user.id=452fcf1ae7a34bb1a2c1010575de2baa&scope.project.id=632d504031cb466299331db2d9184912&role.id=eb6adc880d6b4bbcb153617ac3cd76ff => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:18.450524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8b69c-4020-4c38-ae1f-0b8c0f333a0f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:18.488502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:18.489152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.489430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.489699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.490287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:18.492225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:18.502095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:18.503194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:18.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 456/915] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:18.510110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:18.620528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1a2a2-ac02-41c3-bbb5-059957353e40 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:18.642466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:18.665478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402dde5-5916-4d3c-be06-9d32dab05eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:18.666373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402dde5-5916-4d3c-be06-9d32dab05eff None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.666821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402dde5-5916-4d3c-be06-9d32dab05eff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.667243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402dde5-5916-4d3c-be06-9d32dab05eff None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.668114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402dde5-5916-4d3c-be06-9d32dab05eff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:18.668570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402dde5-5916-4d3c-be06-9d32dab05eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:18.670410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402dde5-5916-4d3c-be06-9d32dab05eff None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:18.675876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 460/916] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:18.683862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:18.794878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538a5fe-10f6-47df-a7e8-0db442af43bf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:18.830504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40454671-4eef-4fcd-acb1-809e54428094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:18.831111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40454671-4eef-4fcd-acb1-809e54428094 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.831391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40454671-4eef-4fcd-acb1-809e54428094 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.831698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40454671-4eef-4fcd-acb1-809e54428094 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.832326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40454671-4eef-4fcd-acb1-809e54428094 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:18.832629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40454671-4eef-4fcd-acb1-809e54428094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:18.834055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40454671-4eef-4fcd-acb1-809e54428094 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:18.836941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7d377-3c51-498b-bb96-d36eb8f0b2ed None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.837471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7d377-3c51-498b-bb96-d36eb8f0b2ed None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.837937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7d377-3c51-498b-bb96-d36eb8f0b2ed None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.839653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 457/917] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 22:50:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:18.840018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 461/918] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:18.845471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd618b-7080-4349-a6f9-96092b309121 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:18.846510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7d377-3c51-498b-bb96-d36eb8f0b2ed None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:18.847066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd618b-7080-4349-a6f9-96092b309121 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:18.847868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd618b-7080-4349-a6f9-96092b309121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:18.863867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:18.867569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-26bd618b-7080-4349-a6f9-96092b309121 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:18.868862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-26bd618b-7080-4349-a6f9-96092b309121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:18.951580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 458/919] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:18.972648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7d377-3c51-498b-bb96-d36eb8f0b2ed None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:19.010666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:19.011697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:19.012230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:19.012720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:19.013594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:19.015175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:19.037551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 462/920] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:19.243604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd618b-7080-4349-a6f9-96092b309121 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:19.352778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd618b-7080-4349-a6f9-96092b309121 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:19.387707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:19.400109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b6779-9958-4873-8b26-fc5f89489e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:19.400761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b6779-9958-4873-8b26-fc5f89489e12 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:19.401040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b6779-9958-4873-8b26-fc5f89489e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:19.401312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b6779-9958-4873-8b26-fc5f89489e12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:19.401896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b6779-9958-4873-8b26-fc5f89489e12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:19.403288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b6779-9958-4873-8b26-fc5f89489e12 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e57b6779-9958-4873-8b26-fc5f89489e12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:19.413131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:19.414218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:19.415079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 459/921] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:19.421391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:19.458915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:19.484827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164646c5-36f2-4c19-83ee-d80fee9137a6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:19.485263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164646c5-36f2-4c19-83ee-d80fee9137a6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:19.485645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164646c5-36f2-4c19-83ee-d80fee9137a6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:19.487334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 460/922] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:19.497280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69cb9-679a-4d8d-90fa-a2d100de6810 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:19.497929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69cb9-679a-4d8d-90fa-a2d100de6810 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:19.498303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69cb9-679a-4d8d-90fa-a2d100de6810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:19.522713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-13a69cb9-679a-4d8d-90fa-a2d100de6810 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:19.523943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-13a69cb9-679a-4d8d-90fa-a2d100de6810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:19.539086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f713ef7-cd64-496d-a738-384303031320 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:19.583582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:19.584734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:19.585281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:19.585783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:19.586625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:19.587372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:19.589755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:19.599525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 461/923] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:19.600338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 463/924] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:19.602729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69cb9-679a-4d8d-90fa-a2d100de6810 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:19.708892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69cb9-679a-4d8d-90fa-a2d100de6810 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:19.732703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:19.745631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:19.746247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:19.746524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:19.746793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:19.747420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:19.747750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:19.749113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:19.754410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 462/925] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:19.760068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:19.867634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:19.871983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a50d-1573-4a11-9f56-5fe131fd00c1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:19.882169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:19.910567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:19.911420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:19.911896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:19.912318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:19.913229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] RBAC: Adding query filter params (interface=public, service_id=62519c4d7fa94205adafe83b780481e8, region_id=RegionOne) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:19.913717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:19.915246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:19.921619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 464/926] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:50:19 2022] GET /identity/v3/endpoints?service_id=62519c4d7fa94205adafe83b780481e8&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:19.929837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:19.989574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55e77-3ef9-4b21-9a84-1680cb7b6ef5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:20.024974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:20.025759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:20.026429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:20.026429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:20.027143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:20.028652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:20.037515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:20.038790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 22:50:20.039956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:20.042246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d76a4-7160-472b-b06f-d17808d00329 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:20.045742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 463/927] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 9 22:50:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:20.046668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:20.084059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:20.085545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:20.086013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:20.086488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:20.087299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:20.088886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:20.096330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 465/928] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:19 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 22:50:20.107864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:20.166721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d1081-66c4-4859-9300-3d65297b1b2b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:20.206507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:20.207480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:20.207971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:20.208427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:20.209250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:20.209761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:20.212010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:20.218209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 464/929] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:20.224746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:20.260244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:20.343849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1739f8-f8c8-4d8e-8ec6-378b4b1fb106 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:20.378916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bcfaa-cdf1-4963-bb06-0fe6cd4ff245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:20.379558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bcfaa-cdf1-4963-bb06-0fe6cd4ff245 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:20.379841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bcfaa-cdf1-4963-bb06-0fe6cd4ff245 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:20.380111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bcfaa-cdf1-4963-bb06-0fe6cd4ff245 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:20.380729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bcfaa-cdf1-4963-bb06-0fe6cd4ff245 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:20.381027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bcfaa-cdf1-4963-bb06-0fe6cd4ff245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:20.383094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bcfaa-cdf1-4963-bb06-0fe6cd4ff245 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:20.388467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 466/930] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:20.393876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:20.503281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e01e0-8b27-4a55-9662-22dbcaca7fb6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:20.518707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:20.540674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64b97e-9aa3-4472-b229-361ff3357c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:20.541611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64b97e-9aa3-4472-b229-361ff3357c69 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:20.542075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64b97e-9aa3-4472-b229-361ff3357c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:20.542549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64b97e-9aa3-4472-b229-361ff3357c69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:20.543427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64b97e-9aa3-4472-b229-361ff3357c69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:20.545411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64b97e-9aa3-4472-b229-361ff3357c69 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:20.568734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 465/931] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:20.722549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fd2a2-4f92-46be-b7b2-0604168c9896 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:20.722883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fd2a2-4f92-46be-b7b2-0604168c9896 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:20.723090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fd2a2-4f92-46be-b7b2-0604168c9896 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:20.724759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 467/932] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:20.730619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4b03-5f02-4a69-9145-c04104540011 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:20.731238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4b03-5f02-4a69-9145-c04104540011 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:20.731734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4b03-5f02-4a69-9145-c04104540011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:20.752667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ba7d4b03-5f02-4a69-9145-c04104540011 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:20.753864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ba7d4b03-5f02-4a69-9145-c04104540011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:20.836379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 466/933] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:21.110480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fd2a2-4f92-46be-b7b2-0604168c9896 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:21.205184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:21.216519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fd2a2-4f92-46be-b7b2-0604168c9896 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:21.251196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:21.251940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:21.252216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:21.252484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:21.264247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:21.269067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:21.269988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:21.270831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:21.274461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 468/934] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 22:50:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:21.276531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4b03-5f02-4a69-9145-c04104540011 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:21.378615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:21.386936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4b03-5f02-4a69-9145-c04104540011 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:21.427259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddff79-3290-4f43-89df-e1b8c41457b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:21.427900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddff79-3290-4f43-89df-e1b8c41457b4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:21.428179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddff79-3290-4f43-89df-e1b8c41457b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:21.428452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddff79-3290-4f43-89df-e1b8c41457b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:21.429727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddff79-3290-4f43-89df-e1b8c41457b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:21.430033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddff79-3290-4f43-89df-e1b8c41457b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:21.431404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddff79-3290-4f43-89df-e1b8c41457b4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:21.440383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 467/935] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 22:50:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:21.445883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:21.499679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b5abd-ae6f-4c3c-a36b-e49f4c4fea5f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:21.499990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b5abd-ae6f-4c3c-a36b-e49f4c4fea5f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:21.500230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b5abd-ae6f-4c3c-a36b-e49f4c4fea5f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:21.501544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 468/936] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:21.510405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc2478-fa42-4363-8331-f7061a442693 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:21.510405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc2478-fa42-4363-8331-f7061a442693 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:21.510405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc2478-fa42-4363-8331-f7061a442693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:21.528895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e0dc2478-fa42-4363-8331-f7061a442693 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:21.530525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e0dc2478-fa42-4363-8331-f7061a442693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:21.556070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b93a0-a0d3-479c-8ade-5395ceb0d902 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:21.593863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:21.594646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:21.594925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:21.595195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:21.611687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:21.611687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:21.612181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 469/937] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:21.623103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:21.624078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:21.624966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:21.624966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:21.624966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:21.628855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 469/938] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:21.630092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc2478-fa42-4363-8331-f7061a442693 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:21.721301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:21.736095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc2478-fa42-4363-8331-f7061a442693 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:21.771614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:21.772224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:21.772563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:21.772832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:21.773526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:21.773827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:21.775194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:21.791652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 470/939] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:21.793607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:21.798367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:21.895662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ae349-ad83-4a82-b100-97354f04ce62 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:21.900452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:21.932231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:21.933025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:21.933306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:21.933573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:21.941967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:21.943409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:21.956135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:21.957212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:21.958167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 470/940] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:50:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:21.964696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.010860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236ab9a-61a6-4ebd-900f-41ba51edae49 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.041381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:22.052589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.053206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.053477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.054266 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.054924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.056469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.059893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING keystone.server.flask.common [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 22:50:22.082345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 22:50:22.082345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 22:50:22.082345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [parameters: {'id': '04b2ea108e97413a9a342853de765f9a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 22:50:22.082345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 22:50:22.087226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca47140-cecd-4856-9e6a-9b761cd29125 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 04b2ea108e97413a9a342853de765f9a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 04b2ea108e97413a9a342853de765f9a. Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 22:50:22.099333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 22:50:22.100220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Sep 09 22:50:22.101312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 22:50:22.102193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 22:50:22.108077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 22:50:22.108890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'id': '04b2ea108e97413a9a342853de765f9a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:22.111465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:22.113349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 04b2ea108e97413a9a342853de765f9a. Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 471/941] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Sep 9 22:50:21 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:22.116904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.119114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:22.141162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.141830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.142036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.142303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.143093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:22.143362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.144887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.170427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 471/942] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 22:50:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:22.171800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.249642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67707a-579a-4894-bf8b-ad15bbecd2bd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.288576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.289437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.289836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.290111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.298439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.300207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:22.313270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:22.314265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:22.315151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 472/943] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:22.320861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b4b0-3cca-45fe-a970-7112f4efe3f8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.321217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.351957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:22.365155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699956b-2154-4581-bca3-23baa5e21d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.365757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699956b-2154-4581-bca3-23baa5e21d72 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.366565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699956b-2154-4581-bca3-23baa5e21d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.366837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699956b-2154-4581-bca3-23baa5e21d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.377730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699956b-2154-4581-bca3-23baa5e21d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.379180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699956b-2154-4581-bca3-23baa5e21d72 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2699956b-2154-4581-bca3-23baa5e21d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:22.394660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:22.395692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:22.396606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:22.396606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:22.396606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:22.399584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 472/944] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:22.400370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:22.402004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba2258-8a9c-49c2-b51b-91aa146421d6 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.402676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba2258-8a9c-49c2-b51b-91aa146421d6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.402985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba2258-8a9c-49c2-b51b-91aa146421d6 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.405778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 473/945] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:22.406126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba2258-8a9c-49c2-b51b-91aa146421d6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.443675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dfc08-d9b1-4eca-909b-88315d1f3802 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.479796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d9ad1-7dd0-4e56-a2be-3be61197a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.480699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d9ad1-7dd0-4e56-a2be-3be61197a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.481178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d9ad1-7dd0-4e56-a2be-3be61197a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.481619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d9ad1-7dd0-4e56-a2be-3be61197a0b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.482454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d9ad1-7dd0-4e56-a2be-3be61197a0b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:22.482823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d9ad1-7dd0-4e56-a2be-3be61197a0b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.484332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d9ad1-7dd0-4e56-a2be-3be61197a0b8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.507873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 473/946] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:22.510182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094b63-676e-4bda-867d-c400bce0c4a9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.511107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094b63-676e-4bda-867d-c400bce0c4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.511850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094b63-676e-4bda-867d-c400bce0c4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.521593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba2258-8a9c-49c2-b51b-91aa146421d6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.539295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ad094b63-676e-4bda-867d-c400bce0c4a9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:22.541745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ad094b63-676e-4bda-867d-c400bce0c4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:22.562402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.563090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.563376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.563689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.564398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:22.564704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.566176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.585492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 474/947] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:22.588140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.634885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:22.641636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 474/948] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:22.707381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9459b5-8da2-4b55-bd8f-f4e06cc5304d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.743727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fba781-add3-4b32-86a4-15440da4b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.744364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fba781-add3-4b32-86a4-15440da4b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.744643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fba781-add3-4b32-86a4-15440da4b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.744912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fba781-add3-4b32-86a4-15440da4b0b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.752228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fba781-add3-4b32-86a4-15440da4b0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.753771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fba781-add3-4b32-86a4-15440da4b0b6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-18fba781-add3-4b32-86a4-15440da4b0b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:22.767684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:22.768677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:22.769561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:22.772594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 475/949] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:22.775255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad094b63-676e-4bda-867d-c400bce0c4a9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.836222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:22.878994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631f0d5-8ebb-4d3d-bfe3-a5842cae65f1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.879327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631f0d5-8ebb-4d3d-bfe3-a5842cae65f1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.879559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631f0d5-8ebb-4d3d-bfe3-a5842cae65f1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.880883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 476/950] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:22.883987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad094b63-676e-4bda-867d-c400bce0c4a9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:22.886128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c1c67-bac4-4200-9d42-6044d5c46caa None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.886410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c1c67-bac4-4200-9d42-6044d5c46caa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.886726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c1c67-bac4-4200-9d42-6044d5c46caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.905703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7f8c1c67-bac4-4200-9d42-6044d5c46caa None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:22.906889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7f8c1c67-bac4-4200-9d42-6044d5c46caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:22.926495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:22.927113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:22.927395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:22.927710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:22.928405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:22.929245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:22.930690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:22.942181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 475/951] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:22.942978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:22.995818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 477/952] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:22.995818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c1c67-bac4-4200-9d42-6044d5c46caa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:23.023512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:23.084339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7f164-95dd-46fe-b7ff-95fb6e0ba12b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.102815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:23.109886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c1c67-bac4-4200-9d42-6044d5c46caa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.130060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.130701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.130978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.131245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.131983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.133312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:23.142250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:23.143248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:23.144330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 476/953] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:23.151715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:23.159599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.160916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.161935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.162437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/452fcf1ae7a34bb1a2c1010575de2baa/roles/eb6adc880d6b4bbcb153617ac3cd76ff` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.165867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=452fcf1ae7a34bb1a2c1010575de2baa, role_id=eb6adc880d6b4bbcb153617ac3cd76ff)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.167851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.185567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 478/954] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:50:22 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/452fcf1ae7a34bb1a2c1010575de2baa/roles/eb6adc880d6b4bbcb153617ac3cd76ff => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:50:23.268313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:23.276449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5db0ad-002c-49b2-9cf2-d38a48da310a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.318137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.320232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.320698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.321218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.322045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:23.322584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.324409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.332220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 477/955] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:23.339259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:23.379689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:23.405019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:23.412008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497a6db-8058-428d-889c-61793dea1d57 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.456566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.457616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.458266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.459075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.460244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.461921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.465307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a030-9f75-41df-a50a-ee768977fc02 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:23.472509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:23.473605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:23.474459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:23.478215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 479/956] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:23.484118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:23.511747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.512638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.513000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.513344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.514408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:23.514949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.516772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.524263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 478/957] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:23.530031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:23.596137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ddab7-317f-4178-98a9-773fa39b7ccf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.633352 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.634858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.635835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.636659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.637629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:23.638145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.640363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.643378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c472d1-d60c-4c3f-8008-d61485a4d3c7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.651932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:23.652595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 480/958] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:23.653781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:23.670480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:23.686813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff411bc-cd4b-4098-80ed-5be7b56a9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.687800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff411bc-cd4b-4098-80ed-5be7b56a9ae8 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.688278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff411bc-cd4b-4098-80ed-5be7b56a9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.688745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff411bc-cd4b-4098-80ed-5be7b56a9ae8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.689696 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff411bc-cd4b-4098-80ed-5be7b56a9ae8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.691323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff411bc-cd4b-4098-80ed-5be7b56a9ae8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.731817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 479/959] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:50:23.785938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d43f-546a-4ba5-881c-b6bbf0154452 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:23.831086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053485c-0660-4aab-9919-61a4146c566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:23.831727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053485c-0660-4aab-9919-61a4146c566a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:23.832002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053485c-0660-4aab-9919-61a4146c566a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:23.832267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053485c-0660-4aab-9919-61a4146c566a None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:23.832888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053485c-0660-4aab-9919-61a4146c566a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:23.833183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053485c-0660-4aab-9919-61a4146c566a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:23.834552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053485c-0660-4aab-9919-61a4146c566a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:23.840704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 481/960] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:24.836062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5319-893b-493c-a32f-43fbf89ab8c5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:24.837039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5319-893b-493c-a32f-43fbf89ab8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:24.837524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a5319-893b-493c-a32f-43fbf89ab8c5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:24.839345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 480/961] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:24.844367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978b764-afef-4f01-8993-ae0932da45c7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:24.844765 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978b764-afef-4f01-8993-ae0932da45c7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:24.844968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978b764-afef-4f01-8993-ae0932da45c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:24.863875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9978b764-afef-4f01-8993-ae0932da45c7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:24.865061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9978b764-afef-4f01-8993-ae0932da45c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:24.939270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 482/962] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:25.221849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5319-893b-493c-a32f-43fbf89ab8c5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:25.332967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a5319-893b-493c-a32f-43fbf89ab8c5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:25.362915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff4de0-dba4-4845-a43d-55023edbd9bd None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:25.363108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff4de0-dba4-4845-a43d-55023edbd9bd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:25.363377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff4de0-dba4-4845-a43d-55023edbd9bd None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:25.366557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 483/963] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:25.369917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:25.370524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138047e0-d421-4b8a-95ef-afd91f479653 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:25.370809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138047e0-d421-4b8a-95ef-afd91f479653 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:25.371078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138047e0-d421-4b8a-95ef-afd91f479653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:25.371417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:25.371970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:25.372524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:25.373370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:25.375151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:25.379506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:25.392209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-138047e0-d421-4b8a-95ef-afd91f479653 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:25.393528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-138047e0-d421-4b8a-95ef-afd91f479653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:25.395357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 22:50:25.395357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 22:50:25.395357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [parameters: {'user_id': '38c75ec4392d4ae089ae4f965b69460f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 22:50:25.395357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 09 22:50:25.403500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 22:50:25.404385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 22:50:25.405340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 22:50:25.406259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 22:50:25.407148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 22:50:25.408031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'user_id': '38c75ec4392d4ae089ae4f965b69460f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:25.408885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 22:50:25.409692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 22:50:25.410608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 22:50:25.410608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 22:50:25.410608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 22:50:25.410608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 22:50:25.410608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 22:50:25.410608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:25.412751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 481/964] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 22:50:25 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:25.419847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:25.456123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:25.470148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 484/965] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:25.544498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302af2d0-301e-4c53-a2b2-0b4812967d1a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:25.581461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:25.582102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:25.582382 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:25.582654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:25.591176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:25.592852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:25.605260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:25.606218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:25.607165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 482/966] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:50:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:25.612433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138047e0-d421-4b8a-95ef-afd91f479653 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:25.718720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138047e0-d421-4b8a-95ef-afd91f479653 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:25.738016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:25.753360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:25.755239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:25.755954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:25.757222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:25.757808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:25.758638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:25.759097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:25.760722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:25.799873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 485/967] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 22:50:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:25.846733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:25.866415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a470-7c9d-47b5-bace-c148b8b3c00e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:25.905871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5108cef-8f7e-4d93-a6bf-bb39da0fff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:25.906547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5108cef-8f7e-4d93-a6bf-bb39da0fff40 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:25.906824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5108cef-8f7e-4d93-a6bf-bb39da0fff40 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:25.907097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5108cef-8f7e-4d93-a6bf-bb39da0fff40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:25.920860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5108cef-8f7e-4d93-a6bf-bb39da0fff40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b5108cef-8f7e-4d93-a6bf-bb39da0fff40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:25.926571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:25.927643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:25.929037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 483/968] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 22:50:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:25.936735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.047434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485b349-66ab-4d90-938a-d44d65e0bfae None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.052881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d68d3-8852-4afe-8931-3500606ec968 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.053109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d68d3-8852-4afe-8931-3500606ec968 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.053341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d68d3-8852-4afe-8931-3500606ec968 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.059300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 484/969] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:26.064764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5633-95be-4c11-9883-f2ecd67500cb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.065161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5633-95be-4c11-9883-f2ecd67500cb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.065434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5633-95be-4c11-9883-f2ecd67500cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.086968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0871f-6d48-4e1c-bebf-3e7994e0c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.087611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0871f-6d48-4e1c-bebf-3e7994e0c98a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.087890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0871f-6d48-4e1c-bebf-3e7994e0c98a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.088157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0871f-6d48-4e1c-bebf-3e7994e0c98a None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.088855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0871f-6d48-4e1c-bebf-3e7994e0c98a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:26.089244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0871f-6d48-4e1c-bebf-3e7994e0c98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.089396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-25ac5633-95be-4c11-9883-f2ecd67500cb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:26.090670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0871f-6d48-4e1c-bebf-3e7994e0c98a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.091010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25ac5633-95be-4c11-9883-f2ecd67500cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:26.101569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 486/970] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 22:50:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:26.102992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e428034-c9d8-4749-90ec-c1a1e2e64340 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.103293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e428034-c9d8-4749-90ec-c1a1e2e64340 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.103579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e428034-c9d8-4749-90ec-c1a1e2e64340 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.104772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 487/971] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:26.108664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e428034-c9d8-4749-90ec-c1a1e2e64340 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.165779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 485/972] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:26.167028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d8c36-03ff-4f4a-a7c1-478d50501a32 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.167314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d8c36-03ff-4f4a-a7c1-478d50501a32 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.167615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d8c36-03ff-4f4a-a7c1-478d50501a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.176200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:26.191820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-da1d8c36-03ff-4f4a-a7c1-478d50501a32 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:26.193013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-da1d8c36-03ff-4f4a-a7c1-478d50501a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:26.219747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e428034-c9d8-4749-90ec-c1a1e2e64340 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.255196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.255860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.256137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.256439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.264720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 486/973] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:26.267646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.269073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:26.283144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:26.284133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:26.285034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:26.285034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:26.285034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:26.285034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 488/974] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:26.290308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d8c36-03ff-4f4a-a7c1-478d50501a32 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.293986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:26.389826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d8c36-03ff-4f4a-a7c1-478d50501a32 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.428314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.430326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.430701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.431089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.431906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:26.432305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.432461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.433858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.450341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 487/975] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:26.451319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.520918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:26.538200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd743fed-a0fb-436a-8c23-c60c12d7a139 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.541233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:26.561765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbcb7c-a3ff-4926-a166-289049635596 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.574669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.575294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.575601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.575876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.576450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.577794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:26.586263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:26.587398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:26.588444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 489/976] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:26.590449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.595505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.596144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.596422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.596692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.604237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.605675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:26.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:26.618316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 488/977] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:26.619295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.692671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ff605-398e-4252-870e-18793bae1e67 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.733858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.734465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.734743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.735012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.735687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.737337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d575d1-b378-4e63-8908-0e4d1d466a23 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.737426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.752429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 490/978] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:50:26 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:26.753628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.768083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:26.793873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:26.794207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.795938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.796273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.796666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.797665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:26.798125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.800386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.807994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 489/979] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:26.814404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.900272 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631927f-9aad-4256-9a1b-f786c24ea186 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.934258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41e651-e245-4a28-a75f-dec577a2b156 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:26.958288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.959292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.960435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.960891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.962266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:26.962907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.965757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.970053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:26.970685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:26.970942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:26.971212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:26.971870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:26.972165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:26.973544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:26.978538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 490/980] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:26.984432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:26.996602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 491/981] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:27.002416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:27.041982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:27.107474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214b84b-87dc-48bd-b558-35712538099f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:27.115329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:27.130300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070983e7-3b88-4a4a-9d8d-b1a1ba2813ed None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:27.146926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:27.148224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:27.148502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:27.148771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:27.149474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] RBAC: Adding query filter params (interface=admin, service_id=62519c4d7fa94205adafe83b780481e8, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:27.149778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:27.151144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:27.160692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 491/982] 10.222.0.17 () {56 vars in 1249 bytes} [Fri Sep 9 22:50:26 2022] GET /identity/v3/endpoints?service_id=62519c4d7fa94205adafe83b780481e8&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:27.162196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:27.172415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:27.173023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:27.173297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:27.173563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:27.244649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:27.245964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:27.271758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb041-0c64-4cf1-87f5-f82505ac9188 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:27.285809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:27.309753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:27.310373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:27.310657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:27.310932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:27.311563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:27.312941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:27.319428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 492/983] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:27 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 22:50:27.336832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:27.355565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 492/984] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:27 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 22:50:27.361165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:27.481805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7230b-3041-436d-9b3e-0e4eb5d84e55 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:27.520479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:27.521126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:27.521391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:27.521660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:27.532530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:27.534086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:27.549135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:27.550135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:27.556065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:27.556065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:27.556065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:27.556065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 493/985] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:27.556877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:27.657704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e454252-37f5-47d2-9995-4071b0a4e40c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:27.692904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:27.693508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:27.693782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:27.694077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:27.694706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:27.695006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:27.696377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:27.707789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 493/986] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:27.713926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:27.748492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:27.826360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87f46f-f183-4c77-b228-1f2c0de0a2aa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:27.860143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:27.860779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:27.861062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:27.861336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:27.868531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:27.869981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:27.881179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:27.882569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:27.888283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 494/987] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:27.890405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:27.937956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:27.992787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4917aa-5de4-4096-a4ef-6433b9087cae None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:28.026363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be875d65-f346-46d3-b285-8649403a8302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:28.027027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be875d65-f346-46d3-b285-8649403a8302 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:28.027313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be875d65-f346-46d3-b285-8649403a8302 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:28.027666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be875d65-f346-46d3-b285-8649403a8302 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:28.028355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be875d65-f346-46d3-b285-8649403a8302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:28.028663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be875d65-f346-46d3-b285-8649403a8302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:28.030087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be875d65-f346-46d3-b285-8649403a8302 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:28.042297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 494/988] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:28.048333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:28.139210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:28.153104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcf1af-f4b3-4571-a028-4b91ab4d719d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:28.191891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ebd47-6c24-42ed-bfbd-e5252a31be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:28.192493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ebd47-6c24-42ed-bfbd-e5252a31be89 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:28.192773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ebd47-6c24-42ed-bfbd-e5252a31be89 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:28.193041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ebd47-6c24-42ed-bfbd-e5252a31be89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:28.193764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ebd47-6c24-42ed-bfbd-e5252a31be89 None admin] RBAC: Adding query filter params (role.id=eb6adc880d6b4bbcb153617ac3cd76ff, scope.project.id=632d504031cb466299331db2d9184912, user.id=452fcf1ae7a34bb1a2c1010575de2baa) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:28.194174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ebd47-6c24-42ed-bfbd-e5252a31be89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:28.195624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ebd47-6c24-42ed-bfbd-e5252a31be89 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:28.201602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 495/989] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:28 2022] GET /identity/v3/role_assignments?user.id=452fcf1ae7a34bb1a2c1010575de2baa&scope.project.id=632d504031cb466299331db2d9184912&role.id=eb6adc880d6b4bbcb153617ac3cd76ff => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:29.012564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77264-0d4e-4caf-9c65-18ea31d77b2d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:29.013093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77264-0d4e-4caf-9c65-18ea31d77b2d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:29.013093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77264-0d4e-4caf-9c65-18ea31d77b2d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:29.014492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 495/990] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:29.019690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254403f-b08e-4e6b-ad18-b5d353f25ae6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:29.020051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254403f-b08e-4e6b-ad18-b5d353f25ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:29.020418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254403f-b08e-4e6b-ad18-b5d353f25ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:29.039753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2254403f-b08e-4e6b-ad18-b5d353f25ae6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:29.040948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2254403f-b08e-4e6b-ad18-b5d353f25ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:29.113159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 496/991] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:29.379214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b77264-0d4e-4caf-9c65-18ea31d77b2d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:29.486796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b77264-0d4e-4caf-9c65-18ea31d77b2d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:29.494919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:29.503280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0699064-51db-48cd-b1d0-436950df9d00 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:29.503663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0699064-51db-48cd-b1d0-436950df9d00 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:29.503937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0699064-51db-48cd-b1d0-436950df9d00 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:29.505412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 497/992] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:29.510125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f14c-d21c-45f7-be87-cd211d6c86b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:29.510404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f14c-d21c-45f7-be87-cd211d6c86b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:29.510672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f14c-d21c-45f7-be87-cd211d6c86b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:29.524014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:29.524650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:29.524930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:29.525204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:29.525807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:29.527156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:29.529266 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-38b0f14c-d21c-45f7-be87-cd211d6c86b5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:29.530512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38b0f14c-d21c-45f7-be87-cd211d6c86b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:29.536028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:29.537006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:29.538472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:29.542655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 496/993] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:29.543723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:29.609379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 498/994] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:29.646207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1297a49-d1ff-42d5-8a6b-1eef2090f5f0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:29.680713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:29.681309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:29.681586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:29.681856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:29.682470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:29.682769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:29.684153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:29.689087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 497/995] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:29.694713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0f14c-d21c-45f7-be87-cd211d6c86b5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:29.708957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:29.807232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0f14c-d21c-45f7-be87-cd211d6c86b5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:29.841376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:29.841985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:29.842263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:29.842536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:29.843162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:29.843463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:29.844852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:29.849947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 499/996] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:29.856465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:29.889101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:29.891842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:29.962939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d5c14-ab22-4acf-b522-28840326497a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:29.973731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:30.001374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:30.001614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebdb2f-35e9-4233-a4c0-ed39d2fa0f1e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:30.001987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.002266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.002536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.003241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] RBAC: Adding query filter params (interface=public, service_id=a5c7c2b2f119411ca32c30e77293838a, region_id=RegionOne) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:30.003588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:30.004952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:30.011394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 498/997] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:50:29 2022] GET /identity/v3/endpoints?service_id=a5c7c2b2f119411ca32c30e77293838a&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:30.016551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:30.043098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:30.043980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.044432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.044705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.045537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:30.046889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:30.056236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:30.057331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 22:50:30.058175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:30.062242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 500/998] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 22:50:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:30.062967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:30.122351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:30.127077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b6b1d-ac03-4152-94e0-1e07927f1709 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:30.164456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d481d-1852-4497-8826-45466c4587ba None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:30.165000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:30.165507 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.165890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.166164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.166880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:30.168300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:30.175032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 499/999] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:30 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 22:50:30.201233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737150f1-74f7-4083-b3ab-db328db4e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:30.201867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737150f1-74f7-4083-b3ab-db328db4e263 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.202148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737150f1-74f7-4083-b3ab-db328db4e263 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.202419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737150f1-74f7-4083-b3ab-db328db4e263 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.203072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737150f1-74f7-4083-b3ab-db328db4e263 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:30.203376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737150f1-74f7-4083-b3ab-db328db4e263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:30.204842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737150f1-74f7-4083-b3ab-db328db4e263 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:30.211188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 501/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:30.217408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:30.333675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0cb53-e3c6-460b-a282-d355d03c3178 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:30.372741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:30.373353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.373638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.373914 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.374540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:30.374844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:30.376239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:30.382592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 500/1001] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 22:50:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:30.387836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe43e37-cbdc-4085-8f76-f8ce272691cf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.387974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:30.388125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe43e37-cbdc-4085-8f76-f8ce272691cf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.388399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe43e37-cbdc-4085-8f76-f8ce272691cf None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.389880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 502/1002] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:30.394703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd4d9-2304-4943-a183-f64888694b84 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.394994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd4d9-2304-4943-a183-f64888694b84 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.395268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd4d9-2304-4943-a183-f64888694b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.402549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:30.415976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a0ebd4d9-2304-4943-a183-f64888694b84 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:30.416574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:30.417363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0ebd4d9-2304-4943-a183-f64888694b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:30.489547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 503/1003] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:30.493822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b81ad-d54c-4221-84c7-137f84ae4d62 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:30.526902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:30.527645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.527951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.528294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.529046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:30.530466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:30.546995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 501/1004] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:50:30.762539 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebd4d9-2304-4943-a183-f64888694b84 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:30.869506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebd4d9-2304-4943-a183-f64888694b84 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:30.905850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:30.913194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:30.913954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:30.914335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:30.914706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] PATH_INFO: `/v3/services/image` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:30.915385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:30.916974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:30.926541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:30.927753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:30.928836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:30.935131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 504/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 9 22:50:30 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:30.936402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:31.022842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:31.042595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccef1bc-3c8f-4b1e-85d6-6572b385d6de None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:31.078025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c56eb-42a7-4ccf-8aee-263c97052db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:31.078656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c56eb-42a7-4ccf-8aee-263c97052db5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:31.078930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c56eb-42a7-4ccf-8aee-263c97052db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:31.079195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c56eb-42a7-4ccf-8aee-263c97052db5 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:31.079860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c56eb-42a7-4ccf-8aee-263c97052db5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:31.080162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c56eb-42a7-4ccf-8aee-263c97052db5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:31.081527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c56eb-42a7-4ccf-8aee-263c97052db5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:31.091527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 502/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:31.092455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:31.193616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364a57a-8426-4fee-b377-8827aeb32f31 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:31.227422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce3e2a-2f61-4613-bf87-e56f57ea8c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:31.228472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce3e2a-2f61-4613-bf87-e56f57ea8c24 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:31.228757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce3e2a-2f61-4613-bf87-e56f57ea8c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:31.229027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce3e2a-2f61-4613-bf87-e56f57ea8c24 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:31.229646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce3e2a-2f61-4613-bf87-e56f57ea8c24 None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:31.229944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce3e2a-2f61-4613-bf87-e56f57ea8c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:31.231307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce3e2a-2f61-4613-bf87-e56f57ea8c24 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:31.243786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 505/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:32.346232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247228c-2cd6-4341-b4d7-f0c62cabf487 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:32.346762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247228c-2cd6-4341-b4d7-f0c62cabf487 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:32.346762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247228c-2cd6-4341-b4d7-f0c62cabf487 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:32.348291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 503/1008] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:32.353997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522627cf-adf9-4e32-a7bc-964051ae39ef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:32.354387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522627cf-adf9-4e32-a7bc-964051ae39ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:32.354547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522627cf-adf9-4e32-a7bc-964051ae39ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:32.372621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-522627cf-adf9-4e32-a7bc-964051ae39ef None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:32.373790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-522627cf-adf9-4e32-a7bc-964051ae39ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:32.429446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:32.444664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 506/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:32.699401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f961feb-7a82-43fc-910f-f6b22cb2c460 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:32.699789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f961feb-7a82-43fc-910f-f6b22cb2c460 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:32.699882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f961feb-7a82-43fc-910f-f6b22cb2c460 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:32.701374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 504/1010] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:32.706321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb9d61-4747-4ac0-a017-48a4172a1b0e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:32.706617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb9d61-4747-4ac0-a017-48a4172a1b0e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:32.706891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb9d61-4747-4ac0-a017-48a4172a1b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:32.713328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f961feb-7a82-43fc-910f-f6b22cb2c460 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:32.730513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ffdb9d61-4747-4ac0-a017-48a4172a1b0e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:32.731888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ffdb9d61-4747-4ac0-a017-48a4172a1b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:32.804365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 507/1011] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:32.826446 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f961feb-7a82-43fc-910f-f6b22cb2c460 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:32.866528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:32.867166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:32.867445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:32.867742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:32.868324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:32.869681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:32.885700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:32.886776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 22:50:32.888113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:32.891808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 505/1012] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:32.892884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb9d61-4747-4ac0-a017-48a4172a1b0e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:32.994522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb9d61-4747-4ac0-a017-48a4172a1b0e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:33.012308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:33.030752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:33.031363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.031670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.031946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.032573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:33.032874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:33.034241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:33.044380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 508/1013] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:33.045444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:33.062179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:33.085315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:33.151244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a8f4-722b-4df0-9803-69260b433a23 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:33.190528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:33.191146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.191420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.191724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.192202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29e7f7-9a45-4de2-a1ac-f2d28a09537b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:33.192396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:33.192656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:33.194561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:33.199798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 506/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 22:50:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:33.205511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:33.232671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:33.233311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.233587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.233852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.235264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:33.236736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:33.238598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 509/1015] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:33.244619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:33.295564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:33.305505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:33.321099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a704a-4f81-4dcc-91cb-511913c51cd0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:33.355758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397f4d8-39f5-497a-9b1f-a84a1fc2edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:33.356429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f4d8-39f5-497a-9b1f-a84a1fc2edb3 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.356634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f4d8-39f5-497a-9b1f-a84a1fc2edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.356905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f4d8-39f5-497a-9b1f-a84a1fc2edb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.357573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397f4d8-39f5-497a-9b1f-a84a1fc2edb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:33.359001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397f4d8-39f5-497a-9b1f-a84a1fc2edb3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:33.361617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eb7eb-202c-4282-9f6a-83ba5b013c14 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:33.380298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 507/1016] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:50:33.388975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfa9e9-dfae-43d9-96fb-2c1a7e93047e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.389300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfa9e9-dfae-43d9-96fb-2c1a7e93047e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.389522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfa9e9-dfae-43d9-96fb-2c1a7e93047e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.391062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 508/1017] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:33.397347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ea32d-3967-40aa-8827-f45061a7d128 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.397770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ea32d-3967-40aa-8827-f45061a7d128 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.398047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ea32d-3967-40aa-8827-f45061a7d128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.417898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:33.418450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ba5ea32d-3967-40aa-8827-f45061a7d128 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:33.418560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.418895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.419156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.419720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ba5ea32d-3967-40aa-8827-f45061a7d128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:33.419894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:33.421615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:33.455699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 510/1018] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:50:33 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:33.502093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 509/1019] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:33 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:33.776882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:33.877012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:33.881595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17827db9-a5e0-45ae-8772-834c7de01ef1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:33.916985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc4c33-ba8a-4737-81db-c8224dff9116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:33.917596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc4c33-ba8a-4737-81db-c8224dff9116 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:33.917872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc4c33-ba8a-4737-81db-c8224dff9116 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:33.918141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc4c33-ba8a-4737-81db-c8224dff9116 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:33.918799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc4c33-ba8a-4737-81db-c8224dff9116 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:33.920437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc4c33-ba8a-4737-81db-c8224dff9116 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:33.936321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 511/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:50:33 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:35.665707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977797-ca31-4521-81cb-f96abb662eda None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:35.666277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977797-ca31-4521-81cb-f96abb662eda None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:35.666277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977797-ca31-4521-81cb-f96abb662eda None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:35.667746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 510/1021] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:35.673667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e383-25c9-43dc-b089-dc1c0acd974a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:35.674103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e383-25c9-43dc-b089-dc1c0acd974a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:35.674242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e383-25c9-43dc-b089-dc1c0acd974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:35.688964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25021a4-499c-459f-889a-fb0665b6056a None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:35.689397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25021a4-499c-459f-889a-fb0665b6056a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:35.689674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25021a4-499c-459f-889a-fb0665b6056a None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:35.691311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 511/1022] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:35.692608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-aab6e383-25c9-43dc-b089-dc1c0acd974a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:35.693863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aab6e383-25c9-43dc-b089-dc1c0acd974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:35.696452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00aa27-10df-4af0-b73a-bc22bf76666d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:35.696735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00aa27-10df-4af0-b73a-bc22bf76666d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:35.697015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00aa27-10df-4af0-b73a-bc22bf76666d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:35.734610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0b00aa27-10df-4af0-b73a-bc22bf76666d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:35.735851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0b00aa27-10df-4af0-b73a-bc22bf76666d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:35.782943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 512/1023] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:35.822942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:35.839887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 512/1024] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:36.063361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e383-25c9-43dc-b089-dc1c0acd974a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.124066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00aa27-10df-4af0-b73a-bc22bf76666d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.150607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:36.168731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e383-25c9-43dc-b089-dc1c0acd974a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.205617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6d17-cceb-469d-80c6-f90ae5794352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.206251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6d17-cceb-469d-80c6-f90ae5794352 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.206526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6d17-cceb-469d-80c6-f90ae5794352 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.206801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6d17-cceb-469d-80c6-f90ae5794352 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.217235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e6d17-cceb-469d-80c6-f90ae5794352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-459e6d17-cceb-469d-80c6-f90ae5794352 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:36.222058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:36.222994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:36.223892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 513/1025] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:36.224586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da13dd-315b-4821-a7da-2a35db51318a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.224586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da13dd-315b-4821-a7da-2a35db51318a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.224948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da13dd-315b-4821-a7da-2a35db51318a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.227533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 514/1026] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:36.230019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da13dd-315b-4821-a7da-2a35db51318a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.231574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00aa27-10df-4af0-b73a-bc22bf76666d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.266518 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cc378-a7d8-4c0b-99d5-0f7b8f92d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.267155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc378-a7d8-4c0b-99d5-0f7b8f92d79e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.267433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc378-a7d8-4c0b-99d5-0f7b8f92d79e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.267760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc378-a7d8-4c0b-99d5-0f7b8f92d79e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.268857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cc378-a7d8-4c0b-99d5-0f7b8f92d79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.270287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cc378-a7d8-4c0b-99d5-0f7b8f92d79e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.272416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 513/1027] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:36.273531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cffe6-c638-4032-9187-90f9704d0755 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.273816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cffe6-c638-4032-9187-90f9704d0755 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.274087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cffe6-c638-4032-9187-90f9704d0755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.291841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-114cffe6-c638-4032-9187-90f9704d0755 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:36.293027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-114cffe6-c638-4032-9187-90f9704d0755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:36.332799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da13dd-315b-4821-a7da-2a35db51318a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.350162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:36.360452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 514/1028] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:36.361233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cffe6-c638-4032-9187-90f9704d0755 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.369874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.370485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.370764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.371034 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.371729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:36.372036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.373402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.382343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 515/1029] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:36.388325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.397478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:36.469827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cffe6-c638-4032-9187-90f9704d0755 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.492211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3035ad8-50c6-409f-8e6f-6de58dec4f44 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.506539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.507145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.507426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.507862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.508511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.509939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.528397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.529242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.529617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.529989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.543647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 515/1030] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:50:36 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:36.555440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.557133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:36.573061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:36.574269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:36.575384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:36.575384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:36.575384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:36.581180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 516/1031] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:36.582485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.631771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.639085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:36.698459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:36.703098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a8833-a4ca-4ec7-a44d-94995d39d6d3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.745472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.746091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.746365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.746635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.747269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:36.747605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.748957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.752885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795a1c6-690c-468b-bc8b-f0e426d88a13 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.762712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 516/1032] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:36.768700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.790994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.791866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.791996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.792180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.792761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.794115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:36.803689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:36.804691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:36.805872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 517/1033] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:36.810386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.880836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99b1db-9c4f-4379-bfe1-0022829649b0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.892125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:36.913087 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525f8ec-739e-451a-a8b4-1187e9ace5e5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:36.919562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.920462 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.920748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.921024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.928581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.930024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.937583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:36.942137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:36.943077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:36.945116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 517/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:36.949457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:36.953318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:36.953923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:36.954200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:36.954469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:36.955093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] RBAC: Adding query filter params (name=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:36.955389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:36.956775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:36.966778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 518/1035] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:36.967662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:37.065631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55c23a-24ce-4540-96a2-c360b42aa055 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.076918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc055d3d-af77-4b51-8d1b-14c275783a33 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.102398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.103019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.103300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.103662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.104395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:37.104733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.106158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:37.112489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.113323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.113782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.114232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.115054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:37.116175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.117842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:37.123107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 519/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:37.124994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 518/1037] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:50:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:37.128904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:37.133465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:37.164157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:37.180274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:37.230956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf410-50c8-49b9-8e7f-728ae4ed1c87 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.252446 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2b0f-b578-4138-a680-ae7b0f24982c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.265446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.266055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.266331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.266631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.267281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] RBAC: Adding query filter params (interface=public, service_id=764b9194aa494eee814173efd7fe76e1, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:37.267822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.268976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:37.274589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 520/1038] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:50:37 2022] GET /identity/v3/endpoints?service_id=764b9194aa494eee814173efd7fe76e1&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:37.280204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:37.290501 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.291111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.291388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.291694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.361328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.363207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:37.387835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4ebe4-0922-41cd-ad74-7868afb897c9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.392583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:37.401912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:37.424440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.424969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.425246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.425512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.426101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.427466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:37.433890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 521/1039] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:50:37 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 22:50:37.488194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 519/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:50:37.494256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:37.599561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8567820-e606-4b9c-b18e-7f2424600178 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.638836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.639476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.639783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.640056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.651170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.652674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:37.667473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:37.668418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:37.669311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:37.669311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:37.669311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:37.669311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 522/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:37.674497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:37.784862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a549129-09a6-4e4b-ac72-5dda5eebc1e5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.811250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:37.820982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.821596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.821875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.822146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.822779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:37.823078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.824472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:37.836585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 520/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:37.841958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:37.873136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:37.944569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed42c5-ba05-4b99-8355-9941f42d1834 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:37.978791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:37.979682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:37.980111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:37.980548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:37.987451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:37.989204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:38.000617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:38.001671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:38.002797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:38.004792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 523/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:38.010492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:38.116503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16814f4a-942e-4d95-b27c-4be5b0274f13 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:38.152760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772a533-3254-40d9-b10b-0d314eb26899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:38.153373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772a533-3254-40d9-b10b-0d314eb26899 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:38.153654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772a533-3254-40d9-b10b-0d314eb26899 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:38.153923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772a533-3254-40d9-b10b-0d314eb26899 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:38.154659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772a533-3254-40d9-b10b-0d314eb26899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:38.154970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772a533-3254-40d9-b10b-0d314eb26899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:38.156482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772a533-3254-40d9-b10b-0d314eb26899 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:38.174520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 521/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:38.175574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:38.269084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:38.280437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72e0a8-1c44-4a84-8d3f-0c826dc4a0d9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:38.314133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0a4b2-4312-4858-a3df-3620ae7fdb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:38.314740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0a4b2-4312-4858-a3df-3620ae7fdb6c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:38.315016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0a4b2-4312-4858-a3df-3620ae7fdb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:38.315283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0a4b2-4312-4858-a3df-3620ae7fdb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:38.316032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0a4b2-4312-4858-a3df-3620ae7fdb6c None admin] RBAC: Adding query filter params (role.id=eb6adc880d6b4bbcb153617ac3cd76ff, scope.project.id=632d504031cb466299331db2d9184912, user.id=786e4e7b1c5147e093693767b229e52b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:38.316337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0a4b2-4312-4858-a3df-3620ae7fdb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:38.317760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0a4b2-4312-4858-a3df-3620ae7fdb6c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:38.324218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 524/1045] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:38 2022] GET /identity/v3/role_assignments?user.id=786e4e7b1c5147e093693767b229e52b&scope.project.id=632d504031cb466299331db2d9184912&role.id=eb6adc880d6b4bbcb153617ac3cd76ff => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:38.764566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4fa56-cf6f-499e-9a6b-619d8366ed9b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:38.764912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4fa56-cf6f-499e-9a6b-619d8366ed9b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:38.765124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4fa56-cf6f-499e-9a6b-619d8366ed9b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:38.766495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 522/1046] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:38.772028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6e53f-63b3-4837-adeb-909af0ef088c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:38.772167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6e53f-63b3-4837-adeb-909af0ef088c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:38.772347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6e53f-63b3-4837-adeb-909af0ef088c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:38.790521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eaf6e53f-63b3-4837-adeb-909af0ef088c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:38.791744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eaf6e53f-63b3-4837-adeb-909af0ef088c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:38.865306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 525/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:39.139085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4fa56-cf6f-499e-9a6b-619d8366ed9b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:39.193025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:39.248100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4fa56-cf6f-499e-9a6b-619d8366ed9b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:39.283050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:39.283716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:39.284001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:39.284275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:39.294336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:39.299395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:39.300436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:39.301281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:39.303811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 523/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:39.307102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6e53f-63b3-4837-adeb-909af0ef088c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:39.390885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:39.407344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6e53f-63b3-4837-adeb-909af0ef088c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:39.441880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554dfe1-0fd7-4118-a4a2-36716d4dde22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:39.442494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554dfe1-0fd7-4118-a4a2-36716d4dde22 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:39.442774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554dfe1-0fd7-4118-a4a2-36716d4dde22 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:39.443047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554dfe1-0fd7-4118-a4a2-36716d4dde22 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:39.444297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554dfe1-0fd7-4118-a4a2-36716d4dde22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:39.444605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554dfe1-0fd7-4118-a4a2-36716d4dde22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:39.445979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554dfe1-0fd7-4118-a4a2-36716d4dde22 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:39.459071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 526/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:50:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:39.460436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:39.569647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15d835-01f8-4684-8b14-fe3707650611 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:39.576627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:39.605390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:39.606023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:39.606304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:39.606579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] PATH_INFO: `/v3/users/3f9f7671e265465aa6f5757bd0b89268` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:39.619879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] RBAC: Authorizing `identity:get_user(user_id=3f9f7671e265465aa6f5757bd0b89268)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:39.621321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:39.623102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 524/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:39 2022] GET /identity/v3/users/3f9f7671e265465aa6f5757bd0b89268 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:39.628379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebf20b-efe4-4ee2-be5a-ab08949ca46f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:39.628647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:39.628750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebf20b-efe4-4ee2-be5a-ab08949ca46f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:39.628911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebf20b-efe4-4ee2-be5a-ab08949ca46f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:39.630344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 527/1051] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:39.635046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae06ef-0708-4d31-b202-b229198417b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:39.635327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae06ef-0708-4d31-b202-b229198417b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:39.635630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae06ef-0708-4d31-b202-b229198417b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:39.653804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-66ae06ef-0708-4d31-b202-b229198417b5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:39.654894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-66ae06ef-0708-4d31-b202-b229198417b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:39.726286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 528/1052] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:39.737214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff5a4b-0043-4ee6-8568-6b520cd33794 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:39.772106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:39.772708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:39.772988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:39.773260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:39.804585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:39.841512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:39.842961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:39.947686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 525/1053] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 22:50:39.949318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae06ef-0708-4d31-b202-b229198417b5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:39.996311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:40.031014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:40.056506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae06ef-0708-4d31-b202-b229198417b5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:40.065853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:40.092935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:40.093573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.093849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.094117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.095250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] RBAC: Authorizing `identity:get_project(project_id=f9a2549b2c584f6e99beade83a7e1d2d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:40.096782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:40.098351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 529/1054] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:50:39 2022] GET /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:40.104843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:40.112222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c565b-7eb7-4fde-ac44-dc402f6a5c90 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:40.150551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:40.151210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.151449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.151927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] PATH_INFO: `/v3/services/image` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.152310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:40.153728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:40.163289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:40.167695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:40.168779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 526/1055] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 9 22:50:39 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:40.171177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:40.218839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f5406-ef0a-4f8f-a7b5-0dada959f60a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:40.257253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:40.257866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.258150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.258423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.259150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d, user.id=3f9f7671e265465aa6f5757bd0b89268) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:40.259459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:40.261016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:40.266706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 530/1056] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:40 2022] GET /identity/v3/role_assignments?user.id=3f9f7671e265465aa6f5757bd0b89268&scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:40.281554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd6f6a-6c4f-4d81-a449-2476ae19752e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:40.312650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:40.322420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419d2dc-1dee-48ae-b285-968fa7dd32b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:40.323057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419d2dc-1dee-48ae-b285-968fa7dd32b1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.323332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419d2dc-1dee-48ae-b285-968fa7dd32b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.323645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419d2dc-1dee-48ae-b285-968fa7dd32b1 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.324291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419d2dc-1dee-48ae-b285-968fa7dd32b1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:40.324589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419d2dc-1dee-48ae-b285-968fa7dd32b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:40.325986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419d2dc-1dee-48ae-b285-968fa7dd32b1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:40.331901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 527/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:40.337630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:40.351318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:40.452461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e0661-dfb0-4293-b5f5-fb388b38e706 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:40.472365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2e84b-e80d-4689-8a51-c0902dae69d8 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.472644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2e84b-e80d-4689-8a51-c0902dae69d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.472842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2e84b-e80d-4689-8a51-c0902dae69d8 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.474169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 528/1058] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:40.479907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9564b6-0ddf-4665-8a24-8a9dc7adddda None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.480218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9564b6-0ddf-4665-8a24-8a9dc7adddda None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.480571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9564b6-0ddf-4665-8a24-8a9dc7adddda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.488598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:40.489213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.489490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.489759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.490975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:40.491290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:40.492739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:40.498457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 531/1059] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:40.501868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dc9564b6-0ddf-4665-8a24-8a9dc7adddda None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:40.503064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dc9564b6-0ddf-4665-8a24-8a9dc7adddda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:40.505187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:40.576664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 529/1060] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:40.583155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:40.609865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413c95-46a1-4a2c-8382-e3784f83603f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:40.643080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:40.643707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.643986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.644254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:40.644916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:40.646271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:40.671526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 532/1061] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:50:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:50:40.846354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9564b6-0ddf-4665-8a24-8a9dc7adddda None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:40.947568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9564b6-0ddf-4665-8a24-8a9dc7adddda None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:40.970093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:40.988495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:40.989175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:40.989380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:40.989605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:41.000369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:41.005162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:41.006709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:41.007960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 530/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 22:50:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:41.013078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:41.114337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445450c-1351-46ec-a64b-9b0d537a34a3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:41.140994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:41.151148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:41.151797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:41.152079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:41.152347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:41.153023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:41.153324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:41.154727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:41.169356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 533/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 22:50:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:41.170975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:41.276035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7319e37-f98f-431a-a581-3485a843715d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:41.311529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:41.312162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:41.312441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:41.312711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:41.322820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:41.324294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:41.338801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:41.340088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:41.340987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:41.340987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:41.340987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:41.343806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 531/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:41.345575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:41.447324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c244c72-4c11-4bc2-8bf1-85f58e264e43 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:41.480935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:41.481546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:41.481825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:41.482095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:41.482736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:41.483038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:41.484425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:41.501398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 534/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:41.502759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:41.534019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:41.607628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e27ad-ec10-4195-8678-f934e791f986 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:41.640343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:41.640970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:41.641252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:41.641524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:41.649322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:41.650750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:41.661731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:41.663265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:41.669366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 532/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:50:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:41.669966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:41.722350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:41.777656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0339-55c9-41d7-a8eb-918829ce4ba6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:41.812664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:41.813208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:41.813487 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:41.813755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:41.814467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:41.814766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:41.816215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:41.835824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 535/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:50:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:41.839557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:41.938397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:41.948811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187f64-8a01-4033-a4a8-0d79be39ac1d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:41.983649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:41.984281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:41.984559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:41.984828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:41.996371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:41.997812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:42.012524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:42.013495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:42.014713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:42.014713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:42.014713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:42.014713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 533/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:42.019356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:42.124334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925f3a-3085-4f14-9080-e0e53a7a1842 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:42.134022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:42.160171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:42.160779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.161058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.161328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.161968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:42.162297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:42.163690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:42.179596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 536/1069] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:42.182966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:42.285972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda740d2-0ef7-4c26-9a27-e1699131147a None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:42.320535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4dd57f-72b5-46de-8bf9-27ea26fd8e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:42.321165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dd57f-72b5-46de-8bf9-27ea26fd8e8a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.321442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dd57f-72b5-46de-8bf9-27ea26fd8e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.321710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dd57f-72b5-46de-8bf9-27ea26fd8e8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.329116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4dd57f-72b5-46de-8bf9-27ea26fd8e8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:42.330593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4dd57f-72b5-46de-8bf9-27ea26fd8e8a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cf4dd57f-72b5-46de-8bf9-27ea26fd8e8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:42.343893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:42.345029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:42.346124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 534/1070] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:42.353477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:42.449494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0a65c-4873-4ae2-9081-72d5d8d6ce8e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.449689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0a65c-4873-4ae2-9081-72d5d8d6ce8e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.449941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0a65c-4873-4ae2-9081-72d5d8d6ce8e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.451421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 535/1071] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:42.456009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552bb4b-a249-453d-ab6c-a2dd5ddaa631 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.456220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c2fc0-0520-40d0-a533-09caa8b2d022 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:42.456324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552bb4b-a249-453d-ab6c-a2dd5ddaa631 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.456545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552bb4b-a249-453d-ab6c-a2dd5ddaa631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.463370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:42.478605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6552bb4b-a249-453d-ab6c-a2dd5ddaa631 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:42.480134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6552bb4b-a249-453d-ab6c-a2dd5ddaa631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:42.495643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:42.496256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.496531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.496800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.497479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:42.497786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:42.499202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:42.511194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 537/1072] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:42.517677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:42.551349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:42.567831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 536/1073] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:42.628803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2adc20-3e22-49c5-b487-387801f1a33d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:42.661861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2782-28af-48dd-bcb4-ae178360b383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:42.663076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2782-28af-48dd-bcb4-ae178360b383 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.663348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2782-28af-48dd-bcb4-ae178360b383 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.663655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2782-28af-48dd-bcb4-ae178360b383 None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/786e4e7b1c5147e093693767b229e52b/roles/eb6adc880d6b4bbcb153617ac3cd76ff` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.665899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2782-28af-48dd-bcb4-ae178360b383 None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=786e4e7b1c5147e093693767b229e52b, role_id=eb6adc880d6b4bbcb153617ac3cd76ff)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:42.667315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2782-28af-48dd-bcb4-ae178360b383 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:42.681090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 538/1074] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:50:42 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/786e4e7b1c5147e093693767b229e52b/roles/eb6adc880d6b4bbcb153617ac3cd76ff => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 22:50:42.851008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552bb4b-a249-453d-ab6c-a2dd5ddaa631 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:42.922421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199656-0094-46bf-ad65-7dcccaf147b2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.923168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199656-0094-46bf-ad65-7dcccaf147b2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.923684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199656-0094-46bf-ad65-7dcccaf147b2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.925382 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 539/1075] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:42.932754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce230-7036-4f51-9499-84c89beb32c2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:42.933439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce230-7036-4f51-9499-84c89beb32c2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:42.933864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce230-7036-4f51-9499-84c89beb32c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:42.953879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-761ce230-7036-4f51-9499-84c89beb32c2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:42.955078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-761ce230-7036-4f51-9499-84c89beb32c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:42.962653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552bb4b-a249-453d-ab6c-a2dd5ddaa631 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:42.995549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:43.001076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.001713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.001992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.002268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.014576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:43.019617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:43.020574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:43.021441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 537/1076] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:43.027162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:43.034247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 540/1077] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:43.129914 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d6cea-c81a-4716-8dfc-95dfe394f246 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:43.165189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.165793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.166076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.166348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.167011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:43.167312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:43.170399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:43.178490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 538/1078] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:43.184143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce230-7036-4f51-9499-84c89beb32c2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:43.204582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:43.294198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce230-7036-4f51-9499-84c89beb32c2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:43.305770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:43.335897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.336531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.336807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.337081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] PATH_INFO: `/v3/users/3f9f7671e265465aa6f5757bd0b89268` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.338265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] RBAC: Authorizing `identity:get_user(user_id=3f9f7671e265465aa6f5757bd0b89268)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:43.339639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:43.341568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 541/1079] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:43 2022] GET /identity/v3/users/3f9f7671e265465aa6f5757bd0b89268 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:43.347117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:43.368500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:43.417847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b98b93-7c12-4c62-9b41-e756f796b6ee None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:43.438157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:43.453805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.454313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204683-0f34-49a6-bd40-00e520787efb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:43.454447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.454673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.454940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.490929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.491590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.491870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.492136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.493245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=f9a2549b2c584f6e99beade83a7e1d2d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:43.494786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:43.501342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 542/1080] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:50:43 2022] GET /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:43.502294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:43.527718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:43.529165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:43.586289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:43.610458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71254-3d05-439f-8921-6d0396aa5ca7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:43.635654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 539/1081] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 22:50:43.639866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:43.648367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.649313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.649732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.650165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d/users/3f9f7671e265465aa6f5757bd0b89268/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.652402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] RBAC: Authorizing `identity:create_grant(project_id=f9a2549b2c584f6e99beade83a7e1d2d, user_id=3f9f7671e265465aa6f5757bd0b89268, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:43.654194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:43.669060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 543/1082] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:50:43 2022] PUT /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d/users/3f9f7671e265465aa6f5757bd0b89268/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 22:50:43.763796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d30d0-72d4-42a7-9567-219a80cc0eb6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:43.800312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7caaa1-dbee-485a-980d-57f63a86a142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.800951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7caaa1-dbee-485a-980d-57f63a86a142 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.801226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7caaa1-dbee-485a-980d-57f63a86a142 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.801496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7caaa1-dbee-485a-980d-57f63a86a142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.813199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7caaa1-dbee-485a-980d-57f63a86a142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:43.814662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7caaa1-dbee-485a-980d-57f63a86a142 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-cc7caaa1-dbee-485a-980d-57f63a86a142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:43.831012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:43.832078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:43.832971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:43.832971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:43.832971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:43.832971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 540/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:43.838529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:43.857410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:43.946531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202fba9-8d09-49c5-a392-49386c79010f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:43.980740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b3e8f-aa35-4beb-bed5-ff7938a5617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:43.981356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3e8f-aa35-4beb-bed5-ff7938a5617e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:43.981637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3e8f-aa35-4beb-bed5-ff7938a5617e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:43.981909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3e8f-aa35-4beb-bed5-ff7938a5617e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:43.982556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3e8f-aa35-4beb-bed5-ff7938a5617e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:43.982859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3e8f-aa35-4beb-bed5-ff7938a5617e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:43.984255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3e8f-aa35-4beb-bed5-ff7938a5617e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:44.003540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 544/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:44.861577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e458c-8af2-439a-888b-d2da3a129287 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:44.862156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e458c-8af2-439a-888b-d2da3a129287 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:44.862156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e458c-8af2-439a-888b-d2da3a129287 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:44.863473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 541/1085] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:44.868772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d37ff-fbd7-4085-bb86-0071731ae8e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:44.869195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d37ff-fbd7-4085-bb86-0071731ae8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:44.869331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d37ff-fbd7-4085-bb86-0071731ae8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:44.888027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fd8d37ff-fbd7-4085-bb86-0071731ae8e9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:44.889210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fd8d37ff-fbd7-4085-bb86-0071731ae8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:44.954402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 545/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:45.219184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e458c-8af2-439a-888b-d2da3a129287 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:45.229088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:45.320511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e458c-8af2-439a-888b-d2da3a129287 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:45.353222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:45.353839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:45.354117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:45.354385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:45.364725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:45.370015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:45.370981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:45.372006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 542/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 22:50:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:45.377236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d37ff-fbd7-4085-bb86-0071731ae8e9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:45.380372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:45.473983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d37ff-fbd7-4085-bb86-0071731ae8e9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:45.507670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:45.508261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:45.508540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:45.508812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:45.509529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:45.509836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:45.511240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:45.518931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 546/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 22:50:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:45.525573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:45.590050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:45.624829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a0249-44ee-4c84-b92a-ca198ac79301 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:45.657327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b21b54-a749-44b4-b88b-a2953562bd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:45.657944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21b54-a749-44b4-b88b-a2953562bd65 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:45.658220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21b54-a749-44b4-b88b-a2953562bd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:45.658487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21b54-a749-44b4-b88b-a2953562bd65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:45.669218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21b54-a749-44b4-b88b-a2953562bd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:45.670905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21b54-a749-44b4-b88b-a2953562bd65 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-47b21b54-a749-44b4-b88b-a2953562bd65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:45.685146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:45.690924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:45.691957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:45.691957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:45.691957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:45.691957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 543/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:50:45.692215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:45.762099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:45.796312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1d8ad-07d7-41c1-92a4-e6c5148903ad None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:45.818231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0fec4-585e-4b76-9e2b-bbf8e21b0e3c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:45.818641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0fec4-585e-4b76-9e2b-bbf8e21b0e3c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:45.818980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0fec4-585e-4b76-9e2b-bbf8e21b0e3c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:45.820426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 544/1090] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:50:45.825222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb66acb-3e8b-43db-bce1-6232b8a45910 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:45.825555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb66acb-3e8b-43db-bce1-6232b8a45910 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:45.825831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb66acb-3e8b-43db-bce1-6232b8a45910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:45.829072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:45.829682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:45.829963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:45.830234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:45.830872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:45.831173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:45.832562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:45.843665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fbb66acb-3e8b-43db-bce1-6232b8a45910 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:45.844186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 547/1091] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:45.849178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fbb66acb-3e8b-43db-bce1-6232b8a45910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:45.850360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:45.910798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 545/1092] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 22:50:45.947793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c866f-8227-468e-95b6-29221f5708d4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:45.972470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:45.982016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:45.982588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:45.982867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:45.983138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:45.990167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:45.992175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:50:46.003525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:46.004619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:46.005576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 548/1093] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 22:50:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:46.010395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb66acb-3e8b-43db-bce1-6232b8a45910 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.059590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.107482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb66acb-3e8b-43db-bce1-6232b8a45910 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.140609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.141202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.141476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.141738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.142438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:46.142733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.144164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.160300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 546/1094] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:46.165959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.177444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.268362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a9ff7-0ad8-4a45-9e08-aa89bd6ac4a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.273869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb113f2-fd5c-41cb-b594-7c059fb3d19e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.281206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.303620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.304254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.304529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.304799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.309377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.309999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.310276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.310547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.320641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:50:46.325473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:46.326512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:50:46.327340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 547/1095] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:46.332399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.361656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.375650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.377101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.430560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692933d-facb-422c-b83a-38f8ab3247a6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.465341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.465935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.466212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.466478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.467126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:46.467424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.468795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.475083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 549/1096] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:50:46.481259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 548/1097] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:46.481391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.482191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.510329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.563664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.579070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659f59-c793-4a52-bee7-688d449960ca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.581158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d858f-eaaf-4de5-83a3-ae4c84cfc0d3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.613510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.614141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.614421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.614695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.614899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.616092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.616370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.616634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] PATH_INFO: `/v3/users/3f9f7671e265465aa6f5757bd0b89268` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.617761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] RBAC: Authorizing `identity:get_user(user_id=3f9f7671e265465aa6f5757bd0b89268)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.619175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.620776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 549/1098] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/users/3f9f7671e265465aa6f5757bd0b89268 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:46.625221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.626337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.626691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:50:46.641232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:46.642393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:50:46.643533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:50:46.643533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:50:46.643533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:46.643533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 550/1099] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:46.647799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.721713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.731618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa72e2-bf7d-4a41-8901-30ebe4226687 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.747072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7283f-8066-4254-8394-790bce8a6116 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.765773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.766379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.766663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.766937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.779966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.780570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.780848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.781119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.781756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:46.782055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.783421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.790726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.796141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 551/1100] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:46.801706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.833797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.835719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.897372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15004af-a0e4-4f45-98fe-b24985cd2fd6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:46.924003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:46.931871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 550/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 22:50:46.932699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:46.933298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:46.933573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:46.933842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:46.937406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:46.940738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:46.942168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:50:46.953834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:50:46.955281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:50:46.956238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 552/1102] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:50:46.960534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:47.018004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:47.033995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df105-dae2-45af-b2ce-c65aad1e0336 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:47.059296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0e3e-f71b-4137-b009-ff8c79bb30c1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:47.067558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:47.068180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:47.069006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:47.069277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:47.070377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] RBAC: Authorizing `identity:get_project(project_id=f9a2549b2c584f6e99beade83a7e1d2d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:47.071901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:47.073496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 551/1103] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:47.079468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:47.096098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:47.096632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:47.096914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:47.097177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:47.097853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:47.098152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:47.099613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:47.110963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 553/1104] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:47.117428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:47.139989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:47.177099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff8772-f8c0-426f-9984-b6abb663e7f4 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:47.210248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363b293-1843-4257-8489-397aa8051b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:47.210854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363b293-1843-4257-8489-397aa8051b98 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:47.211158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363b293-1843-4257-8489-397aa8051b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:47.211433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363b293-1843-4257-8489-397aa8051b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:47.212173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363b293-1843-4257-8489-397aa8051b98 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d, user.id=3f9f7671e265465aa6f5757bd0b89268) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:47.212480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363b293-1843-4257-8489-397aa8051b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:47.213888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363b293-1843-4257-8489-397aa8051b98 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:47.214137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf302559-bdab-47b8-acfa-e73e8de02f7e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:47.219668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 552/1105] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:47 2022] GET /identity/v3/role_assignments?user.id=3f9f7671e265465aa6f5757bd0b89268&scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:47.231937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:47.251990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbed47-8f21-4e55-a3ed-da839e1067be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:47.252618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbed47-8f21-4e55-a3ed-da839e1067be None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:47.252898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbed47-8f21-4e55-a3ed-da839e1067be None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:47.253167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbed47-8f21-4e55-a3ed-da839e1067be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:47.253906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbed47-8f21-4e55-a3ed-da839e1067be None admin] RBAC: Adding query filter params (role.id=eb6adc880d6b4bbcb153617ac3cd76ff, scope.project.id=632d504031cb466299331db2d9184912, user.id=786e4e7b1c5147e093693767b229e52b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:47.254212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbed47-8f21-4e55-a3ed-da839e1067be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:47.255721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbed47-8f21-4e55-a3ed-da839e1067be None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:47.262782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 554/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:47 2022] GET /identity/v3/role_assignments?user.id=786e4e7b1c5147e093693767b229e52b&scope.project.id=632d504031cb466299331db2d9184912&role.id=eb6adc880d6b4bbcb153617ac3cd76ff => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:49.398200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085305-75ff-49fb-af9e-12e5b69d5b9f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:49.398757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085305-75ff-49fb-af9e-12e5b69d5b9f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:49.398757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085305-75ff-49fb-af9e-12e5b69d5b9f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:49.400099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 553/1107] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:49.405468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b619f-f809-4b13-902d-0de3e297284a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:49.405905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b619f-f809-4b13-902d-0de3e297284a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:49.405998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b619f-f809-4b13-902d-0de3e297284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:49.424560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d27b619f-f809-4b13-902d-0de3e297284a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:49.425732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d27b619f-f809-4b13-902d-0de3e297284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:49.488620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 555/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:49.760126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085305-75ff-49fb-af9e-12e5b69d5b9f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:49.858245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085305-75ff-49fb-af9e-12e5b69d5b9f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:49.884063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:49.891998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243a8a3-c1ac-428e-b078-9c85ef9b31dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:49.892610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243a8a3-c1ac-428e-b078-9c85ef9b31dd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:49.892888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243a8a3-c1ac-428e-b078-9c85ef9b31dd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:49.893155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243a8a3-c1ac-428e-b078-9c85ef9b31dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:49.894246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243a8a3-c1ac-428e-b078-9c85ef9b31dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:49.895689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243a8a3-c1ac-428e-b078-9c85ef9b31dd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:49.897254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 554/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:50:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:49.902813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b619f-f809-4b13-902d-0de3e297284a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:49.996828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b619f-f809-4b13-902d-0de3e297284a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:50.009601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:50.030033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db11831-a008-44a2-a14e-8b24b9d2653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:50.030622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db11831-a008-44a2-a14e-8b24b9d2653a None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:50.030896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db11831-a008-44a2-a14e-8b24b9d2653a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:50.031159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db11831-a008-44a2-a14e-8b24b9d2653a None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:50.031829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db11831-a008-44a2-a14e-8b24b9d2653a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:50.033235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db11831-a008-44a2-a14e-8b24b9d2653a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:50.054337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 556/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:50:49 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:50:52.207167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748eab7-5b26-4ae8-9d3d-409bb8bfc583 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:52.207728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748eab7-5b26-4ae8-9d3d-409bb8bfc583 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:52.207728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748eab7-5b26-4ae8-9d3d-409bb8bfc583 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:52.209059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 555/1111] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:52.214259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3abc-8b84-4966-94d4-fad5924f605c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:52.214640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3abc-8b84-4966-94d4-fad5924f605c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:52.214793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3abc-8b84-4966-94d4-fad5924f605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:52.232613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-191a3abc-8b84-4966-94d4-fad5924f605c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:52.233778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-191a3abc-8b84-4966-94d4-fad5924f605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:52.300751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 557/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:52.566127 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748eab7-5b26-4ae8-9d3d-409bb8bfc583 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:52.663874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748eab7-5b26-4ae8-9d3d-409bb8bfc583 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:52.696513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:52.697126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:52.697405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:52.697675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] PATH_INFO: `/v3/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:52.699054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] RBAC: Authorizing `identity:get_role(role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:52.700428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:52.701718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 556/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:52 2022] GET /identity/v3/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:52.707381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a3abc-8b84-4966-94d4-fad5924f605c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:52.800492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:52.804461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a3abc-8b84-4966-94d4-fad5924f605c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:52.839119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:52.839713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:52.839989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:52.840255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] PATH_INFO: `/v3/users/e827b01cb5634de2b15189eaeb8abfd7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:52.853263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] RBAC: Authorizing `identity:get_user(user_id=e827b01cb5634de2b15189eaeb8abfd7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:52.854681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:52.856278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 558/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:52 2022] GET /identity/v3/users/e827b01cb5634de2b15189eaeb8abfd7 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:52.862172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:52.922574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:52.958494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54d065-cf5d-4a6b-bdcb-078f17b3e7dd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:52.992022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:52.992617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:52.992901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:52.993170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:53.057017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:53.058443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:53.115371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:53.152314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 557/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:50:53.158107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:53.252358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eef41-e0ae-436b-ae8e-aa5244a6b546 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:53.285692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad943640-40e9-4f13-bc74-c83a84f34417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:53.286298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad943640-40e9-4f13-bc74-c83a84f34417 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:53.286571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad943640-40e9-4f13-bc74-c83a84f34417 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:53.286862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad943640-40e9-4f13-bc74-c83a84f34417 None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:53.287983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad943640-40e9-4f13-bc74-c83a84f34417 None admin] RBAC: Authorizing `identity:get_project(project_id=f9a2549b2c584f6e99beade83a7e1d2d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:53.289378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad943640-40e9-4f13-bc74-c83a84f34417 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:53.290942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 559/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:50:53 2022] GET /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:53.297322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:53.391138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e014b4-953b-4474-98a7-bfa14286b8b6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:53.423877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affcb59-e30b-4c4e-9b95-b1efd0e0d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:53.424461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affcb59-e30b-4c4e-9b95-b1efd0e0d810 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:53.424738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affcb59-e30b-4c4e-9b95-b1efd0e0d810 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:53.425008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affcb59-e30b-4c4e-9b95-b1efd0e0d810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:53.425711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affcb59-e30b-4c4e-9b95-b1efd0e0d810 None admin] RBAC: Adding query filter params (role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d, user.id=e827b01cb5634de2b15189eaeb8abfd7) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:53.426015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affcb59-e30b-4c4e-9b95-b1efd0e0d810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:53.427991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affcb59-e30b-4c4e-9b95-b1efd0e0d810 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:53.433069 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 558/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:53 2022] GET /identity/v3/role_assignments?user.id=e827b01cb5634de2b15189eaeb8abfd7&scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:55.531837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3b21-d46c-4006-9431-5aab586d1821 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:55.532392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3b21-d46c-4006-9431-5aab586d1821 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:55.532392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3b21-d46c-4006-9431-5aab586d1821 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:55.533715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 560/1118] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:55.539586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39ffd7-db79-4104-bc48-1525a997e3da None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:55.540022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39ffd7-db79-4104-bc48-1525a997e3da None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:55.540114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39ffd7-db79-4104-bc48-1525a997e3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:55.558662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4d39ffd7-db79-4104-bc48-1525a997e3da None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:55.559873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4d39ffd7-db79-4104-bc48-1525a997e3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:55.581492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:55.625412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 559/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:55.891527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3b21-d46c-4006-9431-5aab586d1821 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:55.916437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:55.995804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3b21-d46c-4006-9431-5aab586d1821 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:56.027978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:56.028594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:56.028868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:56.029135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] PATH_INFO: `/v3/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:56.030509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] RBAC: Authorizing `identity:get_role(role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:56.031889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:56.033193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 561/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:55 2022] GET /identity/v3/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:56.039032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39ffd7-db79-4104-bc48-1525a997e3da None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:56.133308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39ffd7-db79-4104-bc48-1525a997e3da None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:56.165501 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:56.166055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:56.166330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:56.166597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] PATH_INFO: `/v3/users/e827b01cb5634de2b15189eaeb8abfd7` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:56.167728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] RBAC: Authorizing `identity:get_user(user_id=e827b01cb5634de2b15189eaeb8abfd7)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:56.169143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:56.170929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 560/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:56 2022] GET /identity/v3/users/e827b01cb5634de2b15189eaeb8abfd7 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:50:56.176527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:56.263001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:56.271621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26453d4-5ff2-461a-8935-daeb4c2db550 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:56.306981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719434cd-d5b4-4651-9573-0ac97cac3dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:56.307643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719434cd-d5b4-4651-9573-0ac97cac3dcb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:56.307925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719434cd-d5b4-4651-9573-0ac97cac3dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:56.308200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719434cd-d5b4-4651-9573-0ac97cac3dcb None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:56.309320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719434cd-d5b4-4651-9573-0ac97cac3dcb None admin] RBAC: Authorizing `identity:get_project(project_id=f9a2549b2c584f6e99beade83a7e1d2d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:56.310765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719434cd-d5b4-4651-9573-0ac97cac3dcb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:56.312434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 562/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:50:56 2022] GET /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:56.318415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:56.344646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:56.414941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdd00e-7b9b-40bf-a1c4-ab6269c9b15b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:56.448690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c18d-f242-4f63-98e3-a2df70cc6a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:56.449269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c18d-f242-4f63-98e3-a2df70cc6a5e None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:56.449546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c18d-f242-4f63-98e3-a2df70cc6a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:56.449815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c18d-f242-4f63-98e3-a2df70cc6a5e None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d/users/e827b01cb5634de2b15189eaeb8abfd7/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:56.451945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0c18d-f242-4f63-98e3-a2df70cc6a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=f9a2549b2c584f6e99beade83a7e1d2d, user_id=e827b01cb5634de2b15189eaeb8abfd7, role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:56.453395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0c18d-f242-4f63-98e3-a2df70cc6a5e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:56.466252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 561/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:50:56 2022] PUT /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d/users/e827b01cb5634de2b15189eaeb8abfd7/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:50:58.578346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8139b-0895-493a-85c2-7c170cb3cd70 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:58.578893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8139b-0895-493a-85c2-7c170cb3cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:58.579254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8139b-0895-493a-85c2-7c170cb3cd70 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:58.581646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 563/1124] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:50:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:50:58.587650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6b2e0-5e4f-4124-8ae8-277ce6debba1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:58.588105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6b2e0-5e4f-4124-8ae8-277ce6debba1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:58.588200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6b2e0-5e4f-4124-8ae8-277ce6debba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:58.605776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-10f6b2e0-5e4f-4124-8ae8-277ce6debba1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:50:58.606954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-10f6b2e0-5e4f-4124-8ae8-277ce6debba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:50:58.670297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 562/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:50:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:50:58.935197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8139b-0895-493a-85c2-7c170cb3cd70 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:59.031479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8139b-0895-493a-85c2-7c170cb3cd70 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:59.063899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:59.064506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:59.064782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:59.065050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] PATH_INFO: `/v3/roles/fa8ccf0556e2436aab8b04e98d8663ca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:59.066406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] RBAC: Authorizing `identity:get_role(role_id=fa8ccf0556e2436aab8b04e98d8663ca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:59.067778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:59.069078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 564/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:58 2022] GET /identity/v3/roles/fa8ccf0556e2436aab8b04e98d8663ca => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:59.074814 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6b2e0-5e4f-4124-8ae8-277ce6debba1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:59.092100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:59.169753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6b2e0-5e4f-4124-8ae8-277ce6debba1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:59.202190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:59.202744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:59.203019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:59.203284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] PATH_INFO: `/v3/users/e827b01cb5634de2b15189eaeb8abfd7` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:59.204525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] RBAC: Authorizing `identity:get_user(user_id=e827b01cb5634de2b15189eaeb8abfd7)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:59.205927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:59.207456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 563/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:50:59 2022] GET /identity/v3/users/e827b01cb5634de2b15189eaeb8abfd7 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:59.213568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:59.240216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:59.310293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06ee1b-7df3-405a-9457-63352d1e4f5b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:59.343560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:59.344153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:59.344428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:59.344696 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:59.408376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:59.409793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:59.436113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:59.504123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 565/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:50:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:50:59.509893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:59.592560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:59.608706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3312415-9365-49bf-9978-e9e48c8963a8 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:59.640618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77512b-d7e7-4bc7-a798-aa71f931bf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:59.641236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77512b-d7e7-4bc7-a798-aa71f931bf87 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:59.641516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77512b-d7e7-4bc7-a798-aa71f931bf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:59.641787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77512b-d7e7-4bc7-a798-aa71f931bf87 None admin] PATH_INFO: `/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:59.642884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77512b-d7e7-4bc7-a798-aa71f931bf87 None admin] RBAC: Authorizing `identity:get_project(project_id=f9a2549b2c584f6e99beade83a7e1d2d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:59.644332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77512b-d7e7-4bc7-a798-aa71f931bf87 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:59.646451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 564/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:50:59 2022] GET /identity/v3/projects/f9a2549b2c584f6e99beade83a7e1d2d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:50:59.652446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:50:59.747595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c728c-a8a8-4acf-8aab-f52aaaea9aac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:50:59.772811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:50:59.780913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32565f-8bfe-47a9-971e-10f8ff4eafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:50:59.781532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32565f-8bfe-47a9-971e-10f8ff4eafc2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:50:59.781809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32565f-8bfe-47a9-971e-10f8ff4eafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:50:59.782076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32565f-8bfe-47a9-971e-10f8ff4eafc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:50:59.782783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32565f-8bfe-47a9-971e-10f8ff4eafc2 None admin] RBAC: Adding query filter params (role.id=fa8ccf0556e2436aab8b04e98d8663ca, scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d, user.id=e827b01cb5634de2b15189eaeb8abfd7) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:50:59.783083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32565f-8bfe-47a9-971e-10f8ff4eafc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:50:59.784524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32565f-8bfe-47a9-971e-10f8ff4eafc2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:50:59.790647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 566/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:50:59 2022] GET /identity/v3/role_assignments?user.id=e827b01cb5634de2b15189eaeb8abfd7&scope.project.id=f9a2549b2c584f6e99beade83a7e1d2d&role.id=fa8ccf0556e2436aab8b04e98d8663ca => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:01.915313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d5fe4-5c0f-4278-b343-5ab94343b5a5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:01.915879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d5fe4-5c0f-4278-b343-5ab94343b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:01.915971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d5fe4-5c0f-4278-b343-5ab94343b5a5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:01.917229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 565/1131] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:01.922595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7624d-bdf6-41c1-a357-9c6269e0f150 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:01.923046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7624d-bdf6-41c1-a357-9c6269e0f150 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:01.923151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7624d-bdf6-41c1-a357-9c6269e0f150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:01.941479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7dc7624d-bdf6-41c1-a357-9c6269e0f150 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:01.943309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7dc7624d-bdf6-41c1-a357-9c6269e0f150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:02.009826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 567/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:02.282844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d5fe4-5c0f-4278-b343-5ab94343b5a5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:02.383864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d5fe4-5c0f-4278-b343-5ab94343b5a5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:02.417059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a78539-064e-41dc-80f1-c5464961ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:02.417679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a78539-064e-41dc-80f1-c5464961ca5d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:02.417961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a78539-064e-41dc-80f1-c5464961ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:02.418231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a78539-064e-41dc-80f1-c5464961ca5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:02.419334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a78539-064e-41dc-80f1-c5464961ca5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:02.421045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a78539-064e-41dc-80f1-c5464961ca5d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:02.422596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 566/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:51:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:02.428900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7624d-bdf6-41c1-a357-9c6269e0f150 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:02.525840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7624d-bdf6-41c1-a357-9c6269e0f150 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:02.548079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:02.559768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6072f30-45e1-49fa-a0e9-046e47204165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:02.560375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6072f30-45e1-49fa-a0e9-046e47204165 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:02.560655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6072f30-45e1-49fa-a0e9-046e47204165 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:02.560929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6072f30-45e1-49fa-a0e9-046e47204165 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:02.561596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6072f30-45e1-49fa-a0e9-046e47204165 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:02.563019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6072f30-45e1-49fa-a0e9-046e47204165 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:02.590972 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 568/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:51:02 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:51:04.735164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116389ab-0eb2-483b-8f4e-bf3af9172b3c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:04.735597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116389ab-0eb2-483b-8f4e-bf3af9172b3c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:04.735744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116389ab-0eb2-483b-8f4e-bf3af9172b3c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:04.737065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 567/1135] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:04.743009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1bbb-5e31-4636-822f-0d19a40f9bb8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:04.743384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1bbb-5e31-4636-822f-0d19a40f9bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:04.743585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed1bbb-5e31-4636-822f-0d19a40f9bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:04.761756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d6ed1bbb-5e31-4636-822f-0d19a40f9bb8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:04.762926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d6ed1bbb-5e31-4636-822f-0d19a40f9bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:04.827246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 569/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:05.091366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116389ab-0eb2-483b-8f4e-bf3af9172b3c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:05.113142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:05.190971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116389ab-0eb2-483b-8f4e-bf3af9172b3c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:05.228339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4206a-0872-49be-85e5-061da7a9f2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:05.229151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4206a-0872-49be-85e5-061da7a9f2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:05.229655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4206a-0872-49be-85e5-061da7a9f2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:05.230167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4206a-0872-49be-85e5-061da7a9f2b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:05.231463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4206a-0872-49be-85e5-061da7a9f2b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:05.232929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4206a-0872-49be-85e5-061da7a9f2b4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:05.234498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 568/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:51:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:05.240380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed1bbb-5e31-4636-822f-0d19a40f9bb8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:05.338676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed1bbb-5e31-4636-822f-0d19a40f9bb8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:05.340993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:05.373304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1bddf-6c8e-425b-956a-9113398afd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:05.373828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1bddf-6c8e-425b-956a-9113398afd66 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:05.374108 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1bddf-6c8e-425b-956a-9113398afd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:05.374379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1bddf-6c8e-425b-956a-9113398afd66 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:05.375039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1bddf-6c8e-425b-956a-9113398afd66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:05.378208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1bddf-6c8e-425b-956a-9113398afd66 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:05.399673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 570/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:51:05 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:51:07.560196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5ade-e415-43de-be7d-f10ac814c670 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:07.560754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5ade-e415-43de-be7d-f10ac814c670 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:07.560754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5ade-e415-43de-be7d-f10ac814c670 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:07.562052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 569/1139] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:07.567224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e89dd-bb11-4ce4-862d-0b637d4459da None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:07.568169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e89dd-bb11-4ce4-862d-0b637d4459da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:07.568300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e89dd-bb11-4ce4-862d-0b637d4459da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:07.586508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1a7e89dd-bb11-4ce4-862d-0b637d4459da None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:07.587717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1a7e89dd-bb11-4ce4-862d-0b637d4459da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:07.651263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 571/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:07.917766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d5ade-e415-43de-be7d-f10ac814c670 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:08.002848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:08.015931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d5ade-e415-43de-be7d-f10ac814c670 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:08.047959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:08.048569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:08.048848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:08.049114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:08.059265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:51:08.064050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:51:08.065106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:51:08.066129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 570/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:51:08.071359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e89dd-bb11-4ce4-862d-0b637d4459da None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:08.157683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:08.168404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e89dd-bb11-4ce4-862d-0b637d4459da None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:08.200948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:08.201538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:08.201813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:08.202082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:08.202730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:08.203027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:08.204411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:08.212622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 572/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:51:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:08.218357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:08.315705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eddc7e-d70b-4906-ad16-01df5e314ec6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:08.347594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:08.348204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:08.348484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:08.349278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] PATH_INFO: `/v3/users/776aa19d830947bb915e156a1823fed2` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:08.361662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] RBAC: Authorizing `identity:get_user(user_id=776aa19d830947bb915e156a1823fed2)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:08.363087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:08.365291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 571/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:51:08 2022] GET /identity/v3/users/776aa19d830947bb915e156a1823fed2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:08.371030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:08.466273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4802d-bc28-48dd-87fd-6112b89d0b02 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:08.511449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:08.512086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:08.512364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:08.512635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:08.533765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:08.594411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:08.595884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:08.720957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 573/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:51:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:51:08.726751 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:08.735435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:08.825064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faa668-d707-4a62-9ddb-63769fe30551 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:08.857647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122162b-b40c-4ef5-80ab-5446db6b4bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:08.858277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122162b-b40c-4ef5-80ab-5446db6b4bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:08.858560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122162b-b40c-4ef5-80ab-5446db6b4bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:08.858835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122162b-b40c-4ef5-80ab-5446db6b4bc6 None admin] PATH_INFO: `/v3/projects/589eaad21d0f418bba98f72cfae2f9a1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:08.859974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122162b-b40c-4ef5-80ab-5446db6b4bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=589eaad21d0f418bba98f72cfae2f9a1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:08.861418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122162b-b40c-4ef5-80ab-5446db6b4bc6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:08.863041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 572/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:51:08 2022] GET /identity/v3/projects/589eaad21d0f418bba98f72cfae2f9a1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:08.869712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:08.921005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:08.966850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c354bc-4b10-419c-818f-cdc72afe96e2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:09.000706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461bf29a-fa30-4374-a282-a612bf3f16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:09.001249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bf29a-fa30-4374-a282-a612bf3f16b0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:09.001531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bf29a-fa30-4374-a282-a612bf3f16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:09.001805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bf29a-fa30-4374-a282-a612bf3f16b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:09.002522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461bf29a-fa30-4374-a282-a612bf3f16b0 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=589eaad21d0f418bba98f72cfae2f9a1, user.id=776aa19d830947bb915e156a1823fed2) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:09.002830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461bf29a-fa30-4374-a282-a612bf3f16b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:09.004284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461bf29a-fa30-4374-a282-a612bf3f16b0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:09.009500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 574/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:51:08 2022] GET /identity/v3/role_assignments?user.id=776aa19d830947bb915e156a1823fed2&scope.project.id=589eaad21d0f418bba98f72cfae2f9a1&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:11.154732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594eac-6c94-422d-82e7-ece773356e07 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:11.155262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594eac-6c94-422d-82e7-ece773356e07 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:11.155262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594eac-6c94-422d-82e7-ece773356e07 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:11.156643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 573/1147] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:11.162359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab431b9f-61e8-4217-8f45-64a6e3071904 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:11.162757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab431b9f-61e8-4217-8f45-64a6e3071904 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:11.162899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab431b9f-61e8-4217-8f45-64a6e3071904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:11.182175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ab431b9f-61e8-4217-8f45-64a6e3071904 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:11.183348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab431b9f-61e8-4217-8f45-64a6e3071904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:11.246768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 575/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:11.515207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83594eac-6c94-422d-82e7-ece773356e07 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:11.589574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:11.615749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83594eac-6c94-422d-82e7-ece773356e07 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:11.651777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:11.652406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:11.652687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:11.652956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:11.662769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:51:11.667562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:51:11.668654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:51:11.669749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 574/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:51:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:51:11.675163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab431b9f-61e8-4217-8f45-64a6e3071904 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:11.719267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:11.771410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab431b9f-61e8-4217-8f45-64a6e3071904 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:11.803877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:11.804471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:11.804746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:11.805015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:11.805666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:11.805963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:11.807313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:11.815002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 576/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:11.820825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:11.917901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c05ea1-4f9c-4f42-95c1-94eb58ff4c59 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:11.950338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:11.950967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:11.951249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:11.951553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] PATH_INFO: `/v3/users/776aa19d830947bb915e156a1823fed2` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:11.952677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=776aa19d830947bb915e156a1823fed2)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:11.954108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:11.955703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 575/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:51:11 2022] GET /identity/v3/users/776aa19d830947bb915e156a1823fed2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:11.961544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:12.058559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47060fb6-ff37-48ef-8b7a-e1a5d870cf64 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:12.066918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:12.091474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c129c-2b63-4f45-9084-19ccb786bbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:12.092139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c129c-2b63-4f45-9084-19ccb786bbbb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:12.092419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c129c-2b63-4f45-9084-19ccb786bbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:12.092692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c129c-2b63-4f45-9084-19ccb786bbbb None admin] PATH_INFO: `/v3/projects/589eaad21d0f418bba98f72cfae2f9a1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:12.093811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c129c-2b63-4f45-9084-19ccb786bbbb None admin] RBAC: Authorizing `identity:get_project(project_id=589eaad21d0f418bba98f72cfae2f9a1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:12.095252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c129c-2b63-4f45-9084-19ccb786bbbb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:12.096924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 577/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:51:11 2022] GET /identity/v3/projects/589eaad21d0f418bba98f72cfae2f9a1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:12.103340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:12.106172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:12.200287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7b730-7b93-40b4-8d87-7bfefbade8a0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:12.232216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb178f0-0fe6-4993-a4c7-8c64e76ffe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:12.232856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb178f0-0fe6-4993-a4c7-8c64e76ffe20 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:12.233131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb178f0-0fe6-4993-a4c7-8c64e76ffe20 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:12.233401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb178f0-0fe6-4993-a4c7-8c64e76ffe20 None admin] PATH_INFO: `/v3/projects/589eaad21d0f418bba98f72cfae2f9a1/users/776aa19d830947bb915e156a1823fed2/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:12.235252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb178f0-0fe6-4993-a4c7-8c64e76ffe20 None admin] RBAC: Authorizing `identity:create_grant(project_id=589eaad21d0f418bba98f72cfae2f9a1, user_id=776aa19d830947bb915e156a1823fed2, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:12.236695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb178f0-0fe6-4993-a4c7-8c64e76ffe20 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:12.249142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 576/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:51:12 2022] PUT /identity/v3/projects/589eaad21d0f418bba98f72cfae2f9a1/users/776aa19d830947bb915e156a1823fed2/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:51:14.397091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2537a-50fd-46d3-b268-b46c5e5f3486 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:14.397628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2537a-50fd-46d3-b268-b46c5e5f3486 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:14.397628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2537a-50fd-46d3-b268-b46c5e5f3486 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:14.398964 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 578/1154] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:14.404348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daacf5-cd1f-4797-bc37-64381bb6987c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:14.404754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daacf5-cd1f-4797-bc37-64381bb6987c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:14.404899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daacf5-cd1f-4797-bc37-64381bb6987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:14.423380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-96daacf5-cd1f-4797-bc37-64381bb6987c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:14.424584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-96daacf5-cd1f-4797-bc37-64381bb6987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:14.478324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:14.488997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 577/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:14.756871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2537a-50fd-46d3-b268-b46c5e5f3486 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:14.855835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2537a-50fd-46d3-b268-b46c5e5f3486 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:14.887467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:14.888112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:14.888392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:14.888663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:14.899658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:51:14.904417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:51:14.905519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:51:14.906567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 579/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:51:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:51:14.912088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daacf5-cd1f-4797-bc37-64381bb6987c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:15.006539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daacf5-cd1f-4797-bc37-64381bb6987c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:15.038892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:15.039479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:15.039784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:15.040053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:15.040696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:15.040991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:15.042340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:15.050212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 578/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:51:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:15.056399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:15.089768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:15.150774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2af8-ef6c-4608-8371-c3f8302c727e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:15.183262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:15.183900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:15.184178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:15.184445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] PATH_INFO: `/v3/users/776aa19d830947bb915e156a1823fed2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:15.185540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] RBAC: Authorizing `identity:get_user(user_id=776aa19d830947bb915e156a1823fed2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:15.186949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:15.188500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 580/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:51:15 2022] GET /identity/v3/users/776aa19d830947bb915e156a1823fed2 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:15.194088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:15.248411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:15.292446 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10c1a5-4a56-400b-a583-3d060be100ac None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:15.325726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:15.326320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:15.326600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:15.326897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:15.390068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:15.391520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:15.443181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:15.487222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 579/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:51:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:51:15.493770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:15.589973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3a2fd-005b-4cb3-b77b-c8ecc523cefb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:15.592156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:15.622178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3d6ad-7b59-4b82-a22a-7978363a3e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:15.622791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3d6ad-7b59-4b82-a22a-7978363a3e31 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:15.623069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3d6ad-7b59-4b82-a22a-7978363a3e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:15.623335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3d6ad-7b59-4b82-a22a-7978363a3e31 None admin] PATH_INFO: `/v3/projects/589eaad21d0f418bba98f72cfae2f9a1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:15.624468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3d6ad-7b59-4b82-a22a-7978363a3e31 None admin] RBAC: Authorizing `identity:get_project(project_id=589eaad21d0f418bba98f72cfae2f9a1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:15.625883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3d6ad-7b59-4b82-a22a-7978363a3e31 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:15.627477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 581/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:51:15 2022] GET /identity/v3/projects/589eaad21d0f418bba98f72cfae2f9a1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:15.634177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:15.728275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9245c-610a-4ec1-a912-c3933a4b19d3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:15.760749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903662c-982f-4cbf-b053-debb47db535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:15.761338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903662c-982f-4cbf-b053-debb47db535e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:15.761614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903662c-982f-4cbf-b053-debb47db535e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:15.761882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903662c-982f-4cbf-b053-debb47db535e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:15.762585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903662c-982f-4cbf-b053-debb47db535e None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=589eaad21d0f418bba98f72cfae2f9a1, user.id=776aa19d830947bb915e156a1823fed2) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:15.762887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903662c-982f-4cbf-b053-debb47db535e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:15.764333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903662c-982f-4cbf-b053-debb47db535e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:15.769803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 580/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:51:15 2022] GET /identity/v3/role_assignments?user.id=776aa19d830947bb915e156a1823fed2&scope.project.id=589eaad21d0f418bba98f72cfae2f9a1&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:17.885715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14426c26-b5fe-42c9-9642-772807511ece None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:17.886264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14426c26-b5fe-42c9-9642-772807511ece None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:17.886264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14426c26-b5fe-42c9-9642-772807511ece None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:17.887587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 582/1162] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:17.892708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492af60e-6588-499c-87a6-d036115be113 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:17.893142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492af60e-6588-499c-87a6-d036115be113 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:17.893265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492af60e-6588-499c-87a6-d036115be113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:17.912078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-492af60e-6588-499c-87a6-d036115be113 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:17.913255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-492af60e-6588-499c-87a6-d036115be113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:17.926249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:17.978092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 581/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:18.250036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14426c26-b5fe-42c9-9642-772807511ece None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:18.348025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14426c26-b5fe-42c9-9642-772807511ece None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:18.379962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:18.380554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:18.380831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:18.381105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:18.420636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:18.447751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:18.449187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:18.542550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 583/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:51:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:51:18.548660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492af60e-6588-499c-87a6-d036115be113 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:18.642861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492af60e-6588-499c-87a6-d036115be113 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:18.675162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e64593-41b8-486a-b5ec-cc1ed46f2dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:18.675801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e64593-41b8-486a-b5ec-cc1ed46f2dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:18.676077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e64593-41b8-486a-b5ec-cc1ed46f2dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:18.676341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e64593-41b8-486a-b5ec-cc1ed46f2dc8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:18.683226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e64593-41b8-486a-b5ec-cc1ed46f2dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:18.684676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e64593-41b8-486a-b5ec-cc1ed46f2dc8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-79e64593-41b8-486a-b5ec-cc1ed46f2dc8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:51:18.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:51:18.696857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:51:18.698084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 582/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:51:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:51:18.702508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:18.769091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:18.797474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2c9fd-ce46-4f2c-baee-224541a9a616 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:18.830105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53a57a-7e30-4226-a78a-40439f073b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:18.830698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53a57a-7e30-4226-a78a-40439f073b57 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:18.830973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53a57a-7e30-4226-a78a-40439f073b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:18.831241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53a57a-7e30-4226-a78a-40439f073b57 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:18.831895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53a57a-7e30-4226-a78a-40439f073b57 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:18.832192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53a57a-7e30-4226-a78a-40439f073b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:18.833637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53a57a-7e30-4226-a78a-40439f073b57 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:18.840724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 584/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 22:51:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:20.949001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af3c0-6c19-4371-8aee-18e9beda380b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:20.949527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af3c0-6c19-4371-8aee-18e9beda380b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:20.949527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041af3c0-6c19-4371-8aee-18e9beda380b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:20.950897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 583/1167] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:20.956054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed4eb5-aa4e-4110-a996-2ee2c05f897f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:20.956471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed4eb5-aa4e-4110-a996-2ee2c05f897f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:20.956596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed4eb5-aa4e-4110-a996-2ee2c05f897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:20.974882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a6ed4eb5-aa4e-4110-a996-2ee2c05f897f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:20.976091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a6ed4eb5-aa4e-4110-a996-2ee2c05f897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:21.046289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 585/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:21.312875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041af3c0-6c19-4371-8aee-18e9beda380b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:21.324524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:21.416897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041af3c0-6c19-4371-8aee-18e9beda380b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:21.448228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99a69d-75b9-42ff-81d6-6e18bf6e53f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:21.448822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99a69d-75b9-42ff-81d6-6e18bf6e53f0 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:21.449097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99a69d-75b9-42ff-81d6-6e18bf6e53f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:21.449362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99a69d-75b9-42ff-81d6-6e18bf6e53f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:21.450030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99a69d-75b9-42ff-81d6-6e18bf6e53f0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:21.451366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99a69d-75b9-42ff-81d6-6e18bf6e53f0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:21.466944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 584/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Fri Sep 9 22:51:21 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 22:51:23.620134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b185a-00d6-4bcb-bc3e-d85ff500f062 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:23.620672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b185a-00d6-4bcb-bc3e-d85ff500f062 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:23.620672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b185a-00d6-4bcb-bc3e-d85ff500f062 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:23.622003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 586/1170] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:23.627147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce2b3b-29f3-43b1-926a-70d9ac4297e4 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:23.627544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce2b3b-29f3-43b1-926a-70d9ac4297e4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:23.627738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce2b3b-29f3-43b1-926a-70d9ac4297e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:23.646153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-21ce2b3b-29f3-43b1-926a-70d9ac4297e4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:23.647336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-21ce2b3b-29f3-43b1-926a-70d9ac4297e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:23.707909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:23.715845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 585/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:23.985135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b185a-00d6-4bcb-bc3e-d85ff500f062 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:24.048253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:24.081694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b185a-00d6-4bcb-bc3e-d85ff500f062 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:24.113589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e03f7-4f12-420a-aaa6-6c2f18df53d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:24.114223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e03f7-4f12-420a-aaa6-6c2f18df53d1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:24.114500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e03f7-4f12-420a-aaa6-6c2f18df53d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:24.114771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e03f7-4f12-420a-aaa6-6c2f18df53d1 None admin] PATH_INFO: `/v3/domains/5507d36aca854ec4beb1b90b4e1340a5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:24.124584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e03f7-4f12-420a-aaa6-6c2f18df53d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5507d36aca854ec4beb1b90b4e1340a5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:24.126014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e03f7-4f12-420a-aaa6-6c2f18df53d1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:24.127571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 587/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Sep 9 22:51:23 2022] GET /identity/v3/domains/5507d36aca854ec4beb1b90b4e1340a5 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:24.133901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce2b3b-29f3-43b1-926a-70d9ac4297e4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:24.228423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce2b3b-29f3-43b1-926a-70d9ac4297e4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:24.261776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee9562-558c-48cd-b799-03382f5525a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:24.262368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee9562-558c-48cd-b799-03382f5525a8 None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:24.262645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee9562-558c-48cd-b799-03382f5525a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:24.262911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee9562-558c-48cd-b799-03382f5525a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:24.263587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee9562-558c-48cd-b799-03382f5525a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:24.264997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee9562-558c-48cd-b799-03382f5525a8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:24.290041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 586/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:51:24 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:51:26.437599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94c1fc-4700-46a2-8d95-4bd46583a0f1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:26.438196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94c1fc-4700-46a2-8d95-4bd46583a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:26.438196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94c1fc-4700-46a2-8d95-4bd46583a0f1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:26.439468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 588/1174] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:26.445227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0556-b336-41cb-ab3a-1361c3d3973b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:26.445652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0556-b336-41cb-ab3a-1361c3d3973b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:26.445759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0556-b336-41cb-ab3a-1361c3d3973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:26.464110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cf4f0556-b336-41cb-ab3a-1361c3d3973b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:26.465284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf4f0556-b336-41cb-ab3a-1361c3d3973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:26.509019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:26.530925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 587/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:26.827764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94c1fc-4700-46a2-8d95-4bd46583a0f1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:26.925344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94c1fc-4700-46a2-8d95-4bd46583a0f1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:26.949309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:26.958398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017641a9-c2a5-4f56-b454-0b77d175c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:26.959058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017641a9-c2a5-4f56-b454-0b77d175c229 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:26.959334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017641a9-c2a5-4f56-b454-0b77d175c229 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:26.959632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017641a9-c2a5-4f56-b454-0b77d175c229 None admin] PATH_INFO: `/v3/domains/5507d36aca854ec4beb1b90b4e1340a5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:26.960776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017641a9-c2a5-4f56-b454-0b77d175c229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5507d36aca854ec4beb1b90b4e1340a5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:26.962194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017641a9-c2a5-4f56-b454-0b77d175c229 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:26.963772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 589/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Sep 9 22:51:26 2022] GET /identity/v3/domains/5507d36aca854ec4beb1b90b4e1340a5 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:26.969489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f0556-b336-41cb-ab3a-1361c3d3973b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:27.068263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f0556-b336-41cb-ab3a-1361c3d3973b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:27.101383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4a28e-a7a2-4c78-a085-84ec45abae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:27.101979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4a28e-a7a2-4c78-a085-84ec45abae5c None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:27.102257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4a28e-a7a2-4c78-a085-84ec45abae5c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:27.102526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4a28e-a7a2-4c78-a085-84ec45abae5c None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:27.103167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4a28e-a7a2-4c78-a085-84ec45abae5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:27.104627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4a28e-a7a2-4c78-a085-84ec45abae5c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:27.124916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 588/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:51:26 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:51:29.301095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26b817-37b5-4397-a81c-3d612f169aff None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:29.301706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26b817-37b5-4397-a81c-3d612f169aff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:29.301706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26b817-37b5-4397-a81c-3d612f169aff None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:29.303038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 590/1178] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:29.308377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59783dfb-973f-435c-8016-4fe1c8357f6b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:29.308826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59783dfb-973f-435c-8016-4fe1c8357f6b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:29.308935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59783dfb-973f-435c-8016-4fe1c8357f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:29.327573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-59783dfb-973f-435c-8016-4fe1c8357f6b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:29.328819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-59783dfb-973f-435c-8016-4fe1c8357f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:29.358046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:29.396577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 589/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:29.663381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26b817-37b5-4397-a81c-3d612f169aff None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:29.762737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26b817-37b5-4397-a81c-3d612f169aff None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:29.794246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:29.794863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:29.795393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:29.795688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:29.806306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:51:29.811095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:51:29.812239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:51:29.813115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 591/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:51:29.818774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59783dfb-973f-435c-8016-4fe1c8357f6b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:29.913033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59783dfb-973f-435c-8016-4fe1c8357f6b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:29.945777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:29.946307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:29.946585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:29.946854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:29.947528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:29.947828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:29.949178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:29.956889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 590/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:29.962685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:30.016697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:30.059573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a69c3-2371-4b42-be99-96e1b898b93a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:30.091404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:30.092046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:30.092321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:30.092589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] PATH_INFO: `/v3/users/4fd987a132b045ada966405d3fd62861` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:30.105462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] RBAC: Authorizing `identity:get_user(user_id=4fd987a132b045ada966405d3fd62861)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:30.106887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:30.108472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 592/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:51:29 2022] GET /identity/v3/users/4fd987a132b045ada966405d3fd62861 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:30.114762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:30.141871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:30.211056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bcaa1-2a65-49cb-ac27-febc8e275e5b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:30.243393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:30.244009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:30.244285 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:30.244553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:30.308629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:30.310058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:30.335345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:30.403060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 591/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:51:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:51:30.408874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:30.504050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7fdd4-5334-406c-9ccd-c9cbd38fb7e6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:30.516410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:30.540508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f8571-581a-4bad-856b-bd8fbc7b08bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:30.541124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f8571-581a-4bad-856b-bd8fbc7b08bf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:30.541396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f8571-581a-4bad-856b-bd8fbc7b08bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:30.541659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f8571-581a-4bad-856b-bd8fbc7b08bf None admin] PATH_INFO: `/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:30.542745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f8571-581a-4bad-856b-bd8fbc7b08bf None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebe1afbc0f4ec78b0d0418d03df02c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:30.544188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f8571-581a-4bad-856b-bd8fbc7b08bf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:30.546044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 593/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:51:30 2022] GET /identity/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:30.552180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:30.645734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd5452-21a3-4c7e-aa2e-1a2e0c22a1d1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:30.671021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:30.678949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd02e6-cc7d-495a-9b02-d1e4a5739e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:30.679555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd02e6-cc7d-495a-9b02-d1e4a5739e94 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:30.679835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd02e6-cc7d-495a-9b02-d1e4a5739e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:30.680104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd02e6-cc7d-495a-9b02-d1e4a5739e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:30.680805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd02e6-cc7d-495a-9b02-d1e4a5739e94 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=d8ebe1afbc0f4ec78b0d0418d03df02c, user.id=4fd987a132b045ada966405d3fd62861) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:30.681109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd02e6-cc7d-495a-9b02-d1e4a5739e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:30.682515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd02e6-cc7d-495a-9b02-d1e4a5739e94 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:30.687649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 592/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:51:30 2022] GET /identity/v3/role_assignments?user.id=4fd987a132b045ada966405d3fd62861&scope.project.id=d8ebe1afbc0f4ec78b0d0418d03df02c&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:32.769709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869280b7-5e5a-4844-a273-2caa09886031 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:32.770649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869280b7-5e5a-4844-a273-2caa09886031 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:32.770749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869280b7-5e5a-4844-a273-2caa09886031 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:32.772097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 594/1186] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:32.777284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb93aa-b2cb-4a5b-8607-649faa86e03f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:32.777696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb93aa-b2cb-4a5b-8607-649faa86e03f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:32.777802 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb93aa-b2cb-4a5b-8607-649faa86e03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:32.796167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-63eb93aa-b2cb-4a5b-8607-649faa86e03f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:32.797335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-63eb93aa-b2cb-4a5b-8607-649faa86e03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:32.861296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 593/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:33.127342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869280b7-5e5a-4844-a273-2caa09886031 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:33.224914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869280b7-5e5a-4844-a273-2caa09886031 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:33.257016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:33.257618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:33.257893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:33.258159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:33.267701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:51:33.272994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:51:33.274103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:51:33.275030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 595/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:51:33.280043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb93aa-b2cb-4a5b-8607-649faa86e03f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:33.376719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb93aa-b2cb-4a5b-8607-649faa86e03f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:33.398740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:33.410224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:33.410820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:33.411102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:33.411374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:33.412056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:33.412356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:33.413864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:33.421904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 594/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:33.429617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:33.469427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:33.525188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6138-b07c-443e-b682-b769afc4243a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:33.558600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:33.559220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:33.559504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:33.559782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] PATH_INFO: `/v3/users/4fd987a132b045ada966405d3fd62861` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:33.560879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] RBAC: Authorizing `identity:get_user(user_id=4fd987a132b045ada966405d3fd62861)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:33.562288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:33.563873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 596/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:51:33 2022] GET /identity/v3/users/4fd987a132b045ada966405d3fd62861 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:33.569667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:33.667032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4580a-e4bc-4a72-98f9-9ac6fad0818b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:33.698852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92049d-11ba-40c5-af90-63179dcffc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:33.699463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92049d-11ba-40c5-af90-63179dcffc68 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:33.699768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92049d-11ba-40c5-af90-63179dcffc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:33.700036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92049d-11ba-40c5-af90-63179dcffc68 None admin] PATH_INFO: `/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:33.701152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92049d-11ba-40c5-af90-63179dcffc68 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebe1afbc0f4ec78b0d0418d03df02c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:33.702560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92049d-11ba-40c5-af90-63179dcffc68 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:33.704175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 595/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:51:33 2022] GET /identity/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:33.710350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:33.804273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a4408-370f-4686-8f6b-5cc667c4241d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:33.813150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:33.838599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e6320-2444-4c39-8c01-b82bfd7849a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:33.839240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e6320-2444-4c39-8c01-b82bfd7849a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:33.839537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e6320-2444-4c39-8c01-b82bfd7849a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:33.839814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e6320-2444-4c39-8c01-b82bfd7849a9 None admin] PATH_INFO: `/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c/users/4fd987a132b045ada966405d3fd62861/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:33.841661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e6320-2444-4c39-8c01-b82bfd7849a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8ebe1afbc0f4ec78b0d0418d03df02c, user_id=4fd987a132b045ada966405d3fd62861, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:33.843072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e6320-2444-4c39-8c01-b82bfd7849a9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:33.855129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 597/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:51:33 2022] PUT /identity/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c/users/4fd987a132b045ada966405d3fd62861/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:51:35.997267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ce442-6c66-40fd-b8aa-111574cabbf8 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:35.997807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ce442-6c66-40fd-b8aa-111574cabbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:35.997807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ce442-6c66-40fd-b8aa-111574cabbf8 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:35.999056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 596/1193] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:51:36.007679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5bd13-09c3-4edb-9336-b8e3515c3865 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:36.007679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5bd13-09c3-4edb-9336-b8e3515c3865 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:36.007679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5bd13-09c3-4edb-9336-b8e3515c3865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:36.023418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-32b5bd13-09c3-4edb-9336-b8e3515c3865 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:51:36.024636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-32b5bd13-09c3-4edb-9336-b8e3515c3865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:51:36.090495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 598/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:51:36.360780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ce442-6c66-40fd-b8aa-111574cabbf8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:36.414719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:36.467029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ce442-6c66-40fd-b8aa-111574cabbf8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:36.500653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:36.501281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:36.501563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:36.501833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:36.512835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:51:36.517718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:51:36.518791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:51:36.520489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 597/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:51:36.524877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5bd13-09c3-4edb-9336-b8e3515c3865 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:36.620521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:36.624738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5bd13-09c3-4edb-9336-b8e3515c3865 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:36.658406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:36.659008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:36.659288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:36.659596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:36.660766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:36.661070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:36.662440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:36.670358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 599/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:36.676786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:36.781456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce35e63-016f-45ca-8411-b5ccd2e04bdf None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:36.800296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:36.822996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:36.823650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:36.823931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:36.824203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] PATH_INFO: `/v3/users/4fd987a132b045ada966405d3fd62861` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:36.825312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] RBAC: Authorizing `identity:get_user(user_id=4fd987a132b045ada966405d3fd62861)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:36.826732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:36.828323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 598/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 22:51:36 2022] GET /identity/v3/users/4fd987a132b045ada966405d3fd62861 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:36.834846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:36.929529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55822a07-499f-4e13-858b-bf781c0eba6a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:36.962438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:36.963035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:36.963317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:36.963622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:36.989607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:37.028567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:37.030010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:37.132773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 600/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:51:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:51:37.138855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:37.234739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df696b73-90c2-49ed-b69c-9c0975e6351d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:37.267359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aae8bd-28d5-41f2-b311-58c56e9b6114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:37.268000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aae8bd-28d5-41f2-b311-58c56e9b6114 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:37.268282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aae8bd-28d5-41f2-b311-58c56e9b6114 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:37.268554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aae8bd-28d5-41f2-b311-58c56e9b6114 None admin] PATH_INFO: `/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:37.270221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aae8bd-28d5-41f2-b311-58c56e9b6114 None admin] RBAC: Authorizing `identity:get_project(project_id=d8ebe1afbc0f4ec78b0d0418d03df02c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:37.271777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aae8bd-28d5-41f2-b311-58c56e9b6114 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:37.273362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 599/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 22:51:37 2022] GET /identity/v3/projects/d8ebe1afbc0f4ec78b0d0418d03df02c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:37.279284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:51:37.335475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:37.374783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddc795-b29a-4359-a141-3432950bd8a3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:51:37.408932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea41f6-f703-4e70-badb-c126dbc39d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:51:37.409531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea41f6-f703-4e70-badb-c126dbc39d30 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:51:37.409808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea41f6-f703-4e70-badb-c126dbc39d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:51:37.410078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea41f6-f703-4e70-badb-c126dbc39d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:51:37.411288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea41f6-f703-4e70-badb-c126dbc39d30 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=d8ebe1afbc0f4ec78b0d0418d03df02c, user.id=4fd987a132b045ada966405d3fd62861) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:51:37.411626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea41f6-f703-4e70-badb-c126dbc39d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:51:37.413039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea41f6-f703-4e70-badb-c126dbc39d30 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:51:37.418612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 601/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:51:37 2022] GET /identity/v3/role_assignments?user.id=4fd987a132b045ada966405d3fd62861&scope.project.id=d8ebe1afbc0f4ec78b0d0418d03df02c&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:51:47.263379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=76 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:51:47.416083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=38 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:33.144291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226e59d-bc59-406c-8625-7298444d7a69 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:33.144927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226e59d-bc59-406c-8625-7298444d7a69 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:33.144927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226e59d-bc59-406c-8625-7298444d7a69 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:33.146222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 600/1201] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:52:33.151870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77aa9-964d-4e46-921c-af758d63d24c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:33.152364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77aa9-964d-4e46-921c-af758d63d24c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:33.152489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77aa9-964d-4e46-921c-af758d63d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:33.172828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2fe77aa9-964d-4e46-921c-af758d63d24c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:52:33.174189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2fe77aa9-964d-4e46-921c-af758d63d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:52:33.248510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 602/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:52:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:52:33.527896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226e59d-bc59-406c-8625-7298444d7a69 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:33.673691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226e59d-bc59-406c-8625-7298444d7a69 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:33.731066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:33.731839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:33.732133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:33.732436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:33.744701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:33.746602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:33.763075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:33.764263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:33.765757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:33.765757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:33.765757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:33.765757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 601/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:52:33.771340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe77aa9-964d-4e46-921c-af758d63d24c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:33.896402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe77aa9-964d-4e46-921c-af758d63d24c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:33.941152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:33.949158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea3ab9-029b-4f96-9a8f-4ee49c56ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:33.949917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea3ab9-029b-4f96-9a8f-4ee49c56ad73 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:33.950207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea3ab9-029b-4f96-9a8f-4ee49c56ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:33.950496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea3ab9-029b-4f96-9a8f-4ee49c56ad73 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:33.951131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea3ab9-029b-4f96-9a8f-4ee49c56ad73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:33.951437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea3ab9-029b-4f96-9a8f-4ee49c56ad73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:33.952848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea3ab9-029b-4f96-9a8f-4ee49c56ad73 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:33.971731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 603/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:33.974999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:34.043778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:34.099481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa0eca-8066-4b41-b7de-b736d25c9921 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:34.145244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620f200-2f54-440b-8b16-f60963e0acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:34.146019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620f200-2f54-440b-8b16-f60963e0acec None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:34.146112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620f200-2f54-440b-8b16-f60963e0acec None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:34.146377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620f200-2f54-440b-8b16-f60963e0acec None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:34.147099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620f200-2f54-440b-8b16-f60963e0acec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:34.148684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620f200-2f54-440b-8b16-f60963e0acec None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:34.173102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 602/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 22:52:33 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:52:36.740468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60b072-873f-4a4a-84d5-3d1c5c34ec62 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:36.741007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60b072-873f-4a4a-84d5-3d1c5c34ec62 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:36.741118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60b072-873f-4a4a-84d5-3d1c5c34ec62 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:36.743824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 604/1206] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:52:36.748073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165f01f-511b-488e-a4a3-fde2c63f1dd8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:36.748544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165f01f-511b-488e-a4a3-fde2c63f1dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:36.748678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165f01f-511b-488e-a4a3-fde2c63f1dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:36.769776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a165f01f-511b-488e-a4a3-fde2c63f1dd8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:52:36.770987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a165f01f-511b-488e-a4a3-fde2c63f1dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:52:36.848676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 603/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:52:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:52:37.158873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60b072-873f-4a4a-84d5-3d1c5c34ec62 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:37.291177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60b072-873f-4a4a-84d5-3d1c5c34ec62 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:37.332419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:37.333164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:37.333513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:37.333725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:37.345323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:52:37.350430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:37.355738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:37.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 605/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:52:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:37.358394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165f01f-511b-488e-a4a3-fde2c63f1dd8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:37.405619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:37.474789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165f01f-511b-488e-a4a3-fde2c63f1dd8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:37.518040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:37.518801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:37.519081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:37.519354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:37.520111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:37.520459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:37.522019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:37.531578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 604/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:52:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:52:37.538320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:37.594858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:37.652073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec32e-d651-4830-85bf-7a3d77264abe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:37.690760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:37.691629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:37.691971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:37.692334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:37.706325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:37.708070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:37.725488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:37.726800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:37.728016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:37.728016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:37.728016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:37.733707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 606/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:37.735667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:37.857599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:37.862430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5bff4-c932-41ed-a19c-190e2d47e087 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:37.909109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:37.909941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:37.910319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:37.910705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:37.911834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:37.912275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:37.914103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:37.934281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 605/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:37.935578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:38.058013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d772f-9230-475c-9c2d-433bd5d91dca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:38.059779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:38.099592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:38.100232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:38.100565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:38.100855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:38.108780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:38.110242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:52:38.124278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:38.125490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:38.126974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:38.130499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 607/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:38.131921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:38.248938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88524543-a0ad-41f4-8911-91bb9caecc8d None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:38.297441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:38.298267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:38.298650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:38.299625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:38.300538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:38.300992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:38.302928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:38.325635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 606/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 22:52:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:38.331436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:38.447158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbe95a-8ef2-4063-8eca-033d580c2b4c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:38.484011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:38.484861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:38.485285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:38.485709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:38.534152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:38.578681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:38.580183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:38.697800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 608/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:52:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:52:38.703244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:38.730969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:38.836523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c9ddf-482b-4711-8683-42fa94726edb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:38.885548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:38.886421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:38.886779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:38.887075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:38.900329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:38.902046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:38.926133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:38.927957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:38.929261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:38.929261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:38.929261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:38.929261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 607/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:52:38.934981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:39.021070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:39.071126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2677-ad35-44ec-a7cf-a4dfa8938080 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:39.116212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:39.116744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:39.117013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:39.117283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:39.118466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:39.118804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:39.120563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:39.136820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 609/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:39.138150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:39.226753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:39.275406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca088df-2707-40fc-9560-3779883a8370 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:39.324604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:39.325411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:39.326201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:39.326201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:39.334851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:39.336570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:52:39.349798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:39.351117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:39.352313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 608/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:52:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:39.356997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:39.491019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90b5b9-2bad-41a5-b426-24d465fe515a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:39.523731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:39.538661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d89ba-372d-49a7-ac8b-217772f9519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:39.539363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d89ba-372d-49a7-ac8b-217772f9519f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:39.539966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d89ba-372d-49a7-ac8b-217772f9519f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:39.540276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d89ba-372d-49a7-ac8b-217772f9519f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:39.541168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d89ba-372d-49a7-ac8b-217772f9519f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:39.541598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d89ba-372d-49a7-ac8b-217772f9519f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:39.543664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d89ba-372d-49a7-ac8b-217772f9519f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:39.557513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 610/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:52:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:39.563559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:39.674749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c703b-f8ed-4971-af72-c20bda2f1689 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:39.706338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:39.713776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edcf35-c5a1-4639-9e0d-f1be992eeecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:39.714566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edcf35-c5a1-4639-9e0d-f1be992eeecd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:39.714831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edcf35-c5a1-4639-9e0d-f1be992eeecd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:39.715161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edcf35-c5a1-4639-9e0d-f1be992eeecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:39.716107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edcf35-c5a1-4639-9e0d-f1be992eeecd None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=bd9193fd644f4ec7a39d9c6ab8a1e525) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:39.716447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edcf35-c5a1-4639-9e0d-f1be992eeecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:39.718144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edcf35-c5a1-4639-9e0d-f1be992eeecd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:39.723884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 609/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:52:39 2022] GET /identity/v3/role_assignments?user.id=bd9193fd644f4ec7a39d9c6ab8a1e525&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:41.938070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4354765-7a9c-42e4-9c06-f242658b6084 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:41.938623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4354765-7a9c-42e4-9c06-f242658b6084 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:41.938623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4354765-7a9c-42e4-9c06-f242658b6084 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:41.940472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 611/1220] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:52:41.946206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba038e-5720-4e7a-87bd-d8487923d317 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:41.946654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba038e-5720-4e7a-87bd-d8487923d317 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:41.946949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba038e-5720-4e7a-87bd-d8487923d317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:41.968716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-88ba038e-5720-4e7a-87bd-d8487923d317 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:52:41.970184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-88ba038e-5720-4e7a-87bd-d8487923d317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:52:42.063519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 610/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:52:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:52:42.334166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4354765-7a9c-42e4-9c06-f242658b6084 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:42.452240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4354765-7a9c-42e4-9c06-f242658b6084 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:42.487997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:42.488615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:42.488890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:42.489165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:42.499431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:52:42.504805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:42.506042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:52:42.507212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:42.510957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 612/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:52:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:42.511939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba038e-5720-4e7a-87bd-d8487923d317 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:42.618490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba038e-5720-4e7a-87bd-d8487923d317 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:42.644554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:42.656046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:42.656660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:42.656940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:42.657212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:42.657871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] RBAC: Adding query filter params (name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:42.658786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:42.660193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:42.673988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 611/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:52:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:52:42.675264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:42.721171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:42.785669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647c50-e435-411e-96d2-7262b6110c68 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:42.822149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:42.822775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:42.823056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:42.823330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:42.834186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:42.835663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:42.850662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:42.851813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:42.852848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:42.852848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:42.852848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:42.852848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 613/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:42.857972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:42.985291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3969-34ae-46b9-bc64-c287f9a7f90b None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:43.031480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:43.032172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:43.032560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:43.032978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:43.034334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:43.034777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:43.036519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:43.051622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 612/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:43.058764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:43.176381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:43.183508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708832f6-5397-4e7b-a1eb-2ec9fa3bb77a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:43.222626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:43.223262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:43.223574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:43.223851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:43.231810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:43.233386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:52:43.245359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:43.246454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:43.247658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 614/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:43.252261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:43.309238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:43.372058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c99da-ba91-4753-9e25-abb1810962bd None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:43.411680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:43.412288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:43.412613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:43.412908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:43.413627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:43.413930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:43.415444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:43.434720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 613/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 22:52:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:43.440824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:43.547773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058c5ee-e11f-4ea0-ad20-f22a3e20ba6d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:43.585939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:43.586577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:43.586859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:43.587147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:43.600057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:43.601555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:43.616853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:43.618525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:43.619715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:43.619715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:43.619715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:43.619715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 615/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:43.624827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:43.733941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237d6e0-5082-4a7b-8ca1-ddab2a32b44c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:43.736955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:43.771628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:43.772232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:43.772512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:43.772781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:43.773438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:43.773738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:43.775236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:43.787754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 614/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:43.793505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:43.803610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:43.903614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668207db-1aab-4830-88e7-caaee0be49fd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:43.941879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:43.942513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:43.942813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:43.943043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:43.950024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:43.951464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:52:43.964338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:43.965304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:43.966192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 616/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:52:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:43.972225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:44.088930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda11f-fee5-473c-8e3b-7141762e7ed6 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:44.127942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e710dd-6a0a-4070-bf83-fb19484f21e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:44.128621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e710dd-6a0a-4070-bf83-fb19484f21e8 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:44.128933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e710dd-6a0a-4070-bf83-fb19484f21e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:44.129248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e710dd-6a0a-4070-bf83-fb19484f21e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:44.130536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e710dd-6a0a-4070-bf83-fb19484f21e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:44.130536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e710dd-6a0a-4070-bf83-fb19484f21e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:44.131894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e710dd-6a0a-4070-bf83-fb19484f21e8 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:44.150476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 615/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:52:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:52:44.151713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:44.222809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:44.258065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74a86-7fd0-47a6-9742-9d863a86e6ed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:44.299453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfd944-f110-4226-a17e-43343d29e80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:44.300198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfd944-f110-4226-a17e-43343d29e80c None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:44.300482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfd944-f110-4226-a17e-43343d29e80c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:44.300752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfd944-f110-4226-a17e-43343d29e80c None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/bd9193fd644f4ec7a39d9c6ab8a1e525/roles/8d7867fc676d4963a98f9eaed5f9f9f6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:44.318220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfd944-f110-4226-a17e-43343d29e80c None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=bd9193fd644f4ec7a39d9c6ab8a1e525, role_id=8d7867fc676d4963a98f9eaed5f9f9f6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:44.319917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfd944-f110-4226-a17e-43343d29e80c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:44.334851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 617/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:52:44 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/bd9193fd644f4ec7a39d9c6ab8a1e525/roles/8d7867fc676d4963a98f9eaed5f9f9f6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:52:46.648363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d810e18-493b-4506-b70f-a2172c8a1c48 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:46.648943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d810e18-493b-4506-b70f-a2172c8a1c48 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:46.648943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d810e18-493b-4506-b70f-a2172c8a1c48 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:46.650224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 616/1233] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:52:46.655421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543126e5-5e54-49e3-a1e0-48bf4cce0cea None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:46.655876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543126e5-5e54-49e3-a1e0-48bf4cce0cea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:46.656150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543126e5-5e54-49e3-a1e0-48bf4cce0cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:46.675680 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-543126e5-5e54-49e3-a1e0-48bf4cce0cea None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:52:46.676896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-543126e5-5e54-49e3-a1e0-48bf4cce0cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:52:46.749787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 618/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:52:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:52:47.021505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d810e18-493b-4506-b70f-a2172c8a1c48 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:47.049294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:47.128519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d810e18-493b-4506-b70f-a2172c8a1c48 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:47.163820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:47.164442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:47.164721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:47.164990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:47.175061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:52:47.179951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:47.180990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:47.181845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 617/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 22:52:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:52:47.187225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543126e5-5e54-49e3-a1e0-48bf4cce0cea None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:47.236701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:47.293766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543126e5-5e54-49e3-a1e0-48bf4cce0cea None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:47.333506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:47.334114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:47.334394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:47.334652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:47.335308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:47.335637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:47.336996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:47.344801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 619/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 22:52:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:47.351097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:47.449004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:47.469645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc4c03-2a77-481c-b142-ea1fe326591e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:47.509395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:47.510074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:47.510361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:47.510638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:47.521681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:47.523124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:47.539891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:47.541216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:47.542317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:47.542317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:47.542317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:47.546923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 618/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:52:47.547853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:47.653250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f76f7-eb76-451f-b688-b215330f1843 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:47.663213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:47.690716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:47.691329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:47.691645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:47.691915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:47.692554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:47.692856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:47.694226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:47.707068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 620/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:52:47.714731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:47.835361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1959a0-b6d5-4fa6-9245-eaff2bf876fa None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:47.875936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:47.877191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:47.877567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:47.877945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:47.880985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:47.887668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:47.889304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:52:47.900746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:47.901776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:47.902831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 619/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:47.908825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:48.022382 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f3b5-d83b-44d0-8c3a-e0001c0f4566 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:48.062379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:48.063234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:48.064140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:48.064140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:48.064841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:48.065442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:48.067464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:48.085571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 621/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 22:52:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:52:48.093758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:48.217010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f620a-07e5-4376-9b84-32e912b748d3 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:48.254357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:48.254980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:48.255343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:48.255695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:48.327298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:48.335116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:48.336809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:48.442299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 620/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:52:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 22:52:48.448311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:48.480336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:48.557834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037674c6-2d1d-4ebc-b61f-745f09a83e79 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:48.592812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:48.593374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:48.593656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:48.593925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:48.604293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:48.605744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:48.620914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:48.622085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:48.623200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:48.623200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:48.623200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:48.623200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 622/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:48.629550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:48.734873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:48.741293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eac1e-61bf-42a1-a61a-4f31eebea874 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:48.779472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:48.780081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:48.780362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:48.780637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:48.781303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:48.781607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:48.782990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:48.795007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 621/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:48.801386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:48.886177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:48.909990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8ae53-803c-455e-b591-fdf7d3544064 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:48.946698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:48.947367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:48.947649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:48.947923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:48.955726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:48.957196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:52:48.970009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:48.971015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:48.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 623/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:52:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:48.977593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:49.087988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b55f0-25c8-4d9b-8560-f09c68cb6b34 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:49.124777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d963-820d-4406-b818-dcd7fddb694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:49.125383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d963-820d-4406-b818-dcd7fddb694a None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:49.125661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d963-820d-4406-b818-dcd7fddb694a None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:49.125930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d963-820d-4406-b818-dcd7fddb694a None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:49.127200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d963-820d-4406-b818-dcd7fddb694a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:49.127524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d963-820d-4406-b818-dcd7fddb694a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:49.129109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d963-820d-4406-b818-dcd7fddb694a None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:49.142821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 622/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:52:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:49.148728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:49.252955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f40dcb-e68d-4997-be52-ca68777602b3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:49.296186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af645f0-3efd-4b8b-961b-8e025cace08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:49.297019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af645f0-3efd-4b8b-961b-8e025cace08e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:49.297339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af645f0-3efd-4b8b-961b-8e025cace08e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:49.297669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af645f0-3efd-4b8b-961b-8e025cace08e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:49.298717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af645f0-3efd-4b8b-961b-8e025cace08e None admin] RBAC: Adding query filter params (role.id=8d7867fc676d4963a98f9eaed5f9f9f6, scope.project.id=632d504031cb466299331db2d9184912, user.id=bd9193fd644f4ec7a39d9c6ab8a1e525) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:49.299238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af645f0-3efd-4b8b-961b-8e025cace08e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:49.301466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af645f0-3efd-4b8b-961b-8e025cace08e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:49.309615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 624/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:52:49 2022] GET /identity/v3/role_assignments?user.id=bd9193fd644f4ec7a39d9c6ab8a1e525&scope.project.id=632d504031cb466299331db2d9184912&role.id=8d7867fc676d4963a98f9eaed5f9f9f6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:51.625628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595354c-d621-4351-9eec-0ea0c9c92f71 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:51.626217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595354c-d621-4351-9eec-0ea0c9c92f71 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:51.626217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595354c-d621-4351-9eec-0ea0c9c92f71 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:51.631548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 623/1247] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:52:51.633963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d3f50-0eee-4ae6-ab87-d30033c58759 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:51.634539 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d3f50-0eee-4ae6-ab87-d30033c58759 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:51.634539 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d3f50-0eee-4ae6-ab87-d30033c58759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:51.641378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:51.654862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-106d3f50-0eee-4ae6-ab87-d30033c58759 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:52:51.656078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-106d3f50-0eee-4ae6-ab87-d30033c58759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:52:51.728102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 625/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:52:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:52:52.015371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595354c-d621-4351-9eec-0ea0c9c92f71 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:52.034859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:52.134380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595354c-d621-4351-9eec-0ea0c9c92f71 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:52.172473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:52.173654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:52.174051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:52.174325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:52.187296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:52:52.194203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:52.195354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:52:52.196452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:52.200636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 624/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:52:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:52.201816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d3f50-0eee-4ae6-ab87-d30033c58759 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:52.331331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d3f50-0eee-4ae6-ab87-d30033c58759 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:52.367756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:52.373314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:52.373923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:52.374202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:52.374472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:52.375133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:52.375434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:52.377419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:52.385905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 626/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 22:52:52.391767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:52.476387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:52.502122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b88b-a9cd-4809-8810-6d672fdf7153 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:52.542742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:52.543381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:52.543688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:52.543959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:52.555583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:52.557035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:52.572839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:52.574024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:52.574923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:52.574923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:52.574923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:52.574923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 625/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:52.580163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:52.684667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdd090-7831-4955-b7e6-6bac79bb59af None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:52.721020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:52.721643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:52.721920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:52.722190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:52.722844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:52.723145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:52.724596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:52.737553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 627/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:52.743391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:52.849719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7fac2-9c21-417e-9ec6-030de8d93e8f None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:52.875857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:52.884344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:52.884972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:52.885256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:52.885524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:52.894355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:52.895893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:52:52.906940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:52.908033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:52.908974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 626/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:52.914185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:52.972192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:53.018760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b0305-a354-402c-8214-3463df2fed98 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:53.057939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:53.058551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:53.058828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:53.059095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:53.059849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:53.060150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:53.061568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:53.080235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 628/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 22:52:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:53.086191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:53.184884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a4e7f-b8c3-4505-b2ec-4dbbdf7c95a1 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:53.216906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:53.217492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:53.217769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:53.218035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:53.273208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:53.287695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:53.289182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:53.386724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 627/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:52:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:52:53.393118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:53.488666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098be694-a717-4710-90a3-dd7d2b78ab54 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:53.500981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:53.521622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:53.522234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:53.522509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:53.522774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:53.532352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:53.534310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:53.548580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:53.549555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:53.550572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:53.550572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:53.550572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:53.550572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 629/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:53.555140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:53.637577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:53.649985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977faf1e-aa72-42bb-be3b-bc07f6719cdb None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:53.682846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:53.683434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:53.683739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:53.684009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:53.684630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:53.684928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:53.686271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:53.696988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 628/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:53.702367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:53.799036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ab6e1-24ae-4409-baeb-b32c80c9a426 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:53.831989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:53.832903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:53.833327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:53.833672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:53.840896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:53.842541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:52:53.853732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:53.854783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:52:53.855907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 630/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:52:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:53.861137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:53.956833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c19359-f655-4106-9f97-5d2c2832c032 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:53.989315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499ec3f-85f7-40be-b873-3a1316d6c040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:53.989903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ec3f-85f7-40be-b873-3a1316d6c040 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:53.990182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ec3f-85f7-40be-b873-3a1316d6c040 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:53.990454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ec3f-85f7-40be-b873-3a1316d6c040 None admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:53.991124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499ec3f-85f7-40be-b873-3a1316d6c040 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:53.991426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499ec3f-85f7-40be-b873-3a1316d6c040 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:53.992853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499ec3f-85f7-40be-b873-3a1316d6c040 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:54.004222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 629/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:52:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:54.010155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:54.035258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:54.106612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c385a5-e3c4-463e-901a-b31cc7e5c992 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:54.141896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104f82a-3958-4218-9d74-de4dd5715302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:54.142498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104f82a-3958-4218-9d74-de4dd5715302 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:54.142774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104f82a-3958-4218-9d74-de4dd5715302 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:54.143041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104f82a-3958-4218-9d74-de4dd5715302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:54.143787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104f82a-3958-4218-9d74-de4dd5715302 None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=632d504031cb466299331db2d9184912, user.id=bd9193fd644f4ec7a39d9c6ab8a1e525) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:54.144091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104f82a-3958-4218-9d74-de4dd5715302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:54.145513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104f82a-3958-4218-9d74-de4dd5715302 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:54.150843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 631/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:52:54 2022] GET /identity/v3/role_assignments?user.id=bd9193fd644f4ec7a39d9c6ab8a1e525&scope.project.id=632d504031cb466299331db2d9184912&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:56.411384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ec71c-3a96-4e94-980d-89e77e0fd021 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:56.411983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ec71c-3a96-4e94-980d-89e77e0fd021 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:56.411983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ec71c-3a96-4e94-980d-89e77e0fd021 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:56.413266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 630/1261] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:52:56.418497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f80b-46d9-46f7-9468-ffa351446ea3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:56.418942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f80b-46d9-46f7-9468-ffa351446ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:56.419060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f80b-46d9-46f7-9468-ffa351446ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:56.437170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ab09f80b-46d9-46f7-9468-ffa351446ea3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:52:56.438358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab09f80b-46d9-46f7-9468-ffa351446ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:52:56.502544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 632/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:52:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:52:56.771351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ec71c-3a96-4e94-980d-89e77e0fd021 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:56.782087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:56.869368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ec71c-3a96-4e94-980d-89e77e0fd021 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:56.902154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:56.902762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:56.903035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:56.903301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:56.913447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:52:56.918139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:56.919194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:56.920655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 631/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:56.925300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f80b-46d9-46f7-9468-ffa351446ea3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:56.943346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:57.022391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f80b-46d9-46f7-9468-ffa351446ea3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:57.055971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:57.056670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:57.057021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:57.057347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:57.058001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:57.058303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:57.059692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:57.068015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 633/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:57.073874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:57.139463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:57.173880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069eb6e-9889-401c-96a9-9d7fa6a11dff None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:57.206192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:57.206794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:57.207070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:57.207338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:57.217428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:57.218843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:57.232359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:57.233476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:57.234401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:57.234401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:57.234401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:57.234401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 632/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:57.239168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:57.331649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:57.353315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac22f-de21-413d-89e7-9960ea49b275 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:57.385966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:57.386563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:57.386839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:57.387107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:57.387759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:57.388058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:57.389416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:57.400517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 634/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:57.406343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:57.502549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b427695-0212-4b06-8093-ea02b8339939 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:57.521583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:57.535697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:57.536310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:57.536588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:57.536856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:57.543978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:57.545661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:52:57.556459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:57.557521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:57.558643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 633/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:57.563233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:57.660333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4346999-84e9-4aaa-8436-55ce3f42baab None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:57.692995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:57.693590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:57.693869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:57.694137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:57.694851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:57.695154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:57.696603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:57.705565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:57.714185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 635/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 22:52:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:57.719867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:57.822310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3f552-72af-44fc-a9b0-e125651106b7 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:57.854709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:57.855330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:57.855644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:57.855918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:57.866165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:57.867627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:52:57.881888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:57.882912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:52:57.884291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:52:57.884291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:52:57.884291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:57.884291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 634/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:52:57.888726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:57.984319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786f44e-8797-4f3b-adbf-0166a79b7afb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:58.016391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:58.017543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:58.017817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:58.018083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:58.018704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:58.018998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:58.020371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:58.031001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 636/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:58.036513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:58.068182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:58.136851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca3f32-e9f5-4cc0-a517-bc5cec5754b9 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:58.168935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:58.169549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:58.169823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:58.170090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:58.177181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:58.178595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:52:58.190662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:52:58.191730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:52:58.192664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 635/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:52:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 22:52:58.197596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:58.254171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:58.296602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54dd0a-e729-40a1-bb7b-0c50c9b7022c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:58.329339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43febe-d3b5-42ef-b680-1cbddb878e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:58.329930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43febe-d3b5-42ef-b680-1cbddb878e6f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:58.330204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43febe-d3b5-42ef-b680-1cbddb878e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:58.330468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43febe-d3b5-42ef-b680-1cbddb878e6f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:58.331141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43febe-d3b5-42ef-b680-1cbddb878e6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:52:58.331439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43febe-d3b5-42ef-b680-1cbddb878e6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:58.332903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43febe-d3b5-42ef-b680-1cbddb878e6f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:58.345131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 637/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:52:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:52:58.351048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:52:58.445049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:52:58.455460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de42eb-1248-4e50-98d0-b6b4211d0946 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:52:58.489785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea4922-f738-4cc0-a085-e582726b80d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:52:58.490440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea4922-f738-4cc0-a085-e582726b80d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:52:58.490716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea4922-f738-4cc0-a085-e582726b80d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:52:58.490986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea4922-f738-4cc0-a085-e582726b80d1 None admin] PATH_INFO: `/v3/projects/632d504031cb466299331db2d9184912/users/bd9193fd644f4ec7a39d9c6ab8a1e525/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:52:58.492889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea4922-f738-4cc0-a085-e582726b80d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=632d504031cb466299331db2d9184912, user_id=bd9193fd644f4ec7a39d9c6ab8a1e525, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:52:58.494312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea4922-f738-4cc0-a085-e582726b80d1 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:52:58.506488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 636/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 22:52:58 2022] PUT /identity/v3/projects/632d504031cb466299331db2d9184912/users/bd9193fd644f4ec7a39d9c6ab8a1e525/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:53:00.605017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682496-18cb-40f2-8201-1c77b9420ca0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:00.605570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682496-18cb-40f2-8201-1c77b9420ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:00.605570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682496-18cb-40f2-8201-1c77b9420ca0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:00.606901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 638/1274] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:53:00.612758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772b495-c86a-4930-985b-073864035fe5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:00.613140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772b495-c86a-4930-985b-073864035fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:00.613298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772b495-c86a-4930-985b-073864035fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:00.631590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5772b495-c86a-4930-985b-073864035fe5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:53:00.632759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5772b495-c86a-4930-985b-073864035fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:53:00.696999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 637/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:53:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:53:00.964839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d682496-18cb-40f2-8201-1c77b9420ca0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:01.062785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d682496-18cb-40f2-8201-1c77b9420ca0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:01.079953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:01.096877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:01.097501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:01.097778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:01.098046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:01.107885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 22:53:01.113135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:01.114192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:01.115133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 639/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 22:53:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:01.120135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772b495-c86a-4930-985b-073864035fe5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:01.203037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:01.221722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772b495-c86a-4930-985b-073864035fe5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:01.255236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:01.255869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:01.256176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:01.256450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:01.257110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:01.257411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:01.258768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:01.268074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 638/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 22:53:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:01.274312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:01.383783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2e0db-245e-47a1-a14e-a44a186c9189 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:01.417228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:01.417855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:01.418138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:01.418412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:01.428410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:01.429848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:53:01.444492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:01.445822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:53:01.447055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:53:01.447055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:53:01.447055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:01.447055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 640/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:01.451345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:01.554962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350fce9-8be8-49f9-adec-c3f56a2f622e None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:01.587673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:01.588278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:01.588562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:01.588838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:01.589472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:01.589776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:01.591137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:01.602584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 639/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:01.608288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:01.638540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:01.706820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035444f-4791-48a2-bcb4-0d7d945d0887 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:01.739129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:01.739789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:01.740072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:01.740344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:01.747621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:01.749056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 22:53:01.760605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:01.761722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:01.762854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 641/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:53:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:01.768071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:01.771833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:01.879585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6353-952f-4518-8168-1941589f17f0 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:01.911762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:01.912358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:01.912635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:01.912906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:01.913616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:01.913915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:01.915328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:01.932970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 640/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 22:53:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:01.939561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:02.026087 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:02.035473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ef03d-4f6a-4e2a-8c09-e1e2818f6cdd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:02.068288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:02.068877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:02.069154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:02.069424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:02.134076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:02.135454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:02.222630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:02.232677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 642/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 22:53:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 22:53:02.239176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:02.300125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:02.337355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95297c7-e5c2-4b9f-b0ba-7d4d34ba6f39 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:02.370111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:02.370723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:02.370999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:02.371262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:02.381044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:02.382457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 22:53:02.396453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:02.397406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 22:53:02.398405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 22:53:02.398405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 22:53:02.398405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:53:02.398405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 641/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:02.403186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:02.500247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87d834-272f-467f-8e19-ee8b4f89581e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:02.534050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:02.534650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:02.534925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:02.535192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:02.535843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:02.536141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:02.537486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:02.548908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 643/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 22:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:02.554551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:02.649946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07445360-c4ee-4a17-9422-1bfdd252ee3c None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:02.662763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:02.683218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:02.683856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:02.684134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:02.684407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:02.691356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:02.692806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 22:53:02.703642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:02.704673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: ERROR keystone.server.flask.application Sep 09 22:53:02.705792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 642/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 22:53:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:02.710578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:02.761254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:02.809267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7ffd-af59-4214-aa7b-72d577cf5dc2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:02.841175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf0122-39cf-47f7-bb7f-afc3204b3b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:02.841774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf0122-39cf-47f7-bb7f-afc3204b3b48 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:02.842053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf0122-39cf-47f7-bb7f-afc3204b3b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:02.842322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf0122-39cf-47f7-bb7f-afc3204b3b48 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:02.842999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf0122-39cf-47f7-bb7f-afc3204b3b48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:02.843302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf0122-39cf-47f7-bb7f-afc3204b3b48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:02.844785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf0122-39cf-47f7-bb7f-afc3204b3b48 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:02.856210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 644/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 22:53:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:02.862758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:02.959949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f3f6b-2d0d-4654-930e-209682e5ef30 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:02.992488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9368-aea7-4dc2-8741-b011d033a74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:02.993067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9368-aea7-4dc2-8741-b011d033a74b None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:02.993341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9368-aea7-4dc2-8741-b011d033a74b None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:02.993608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9368-aea7-4dc2-8741-b011d033a74b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:02.994311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9368-aea7-4dc2-8741-b011d033a74b None admin] RBAC: Adding query filter params (role.id=67db52226a7a4fd0b469e8de2c12b8e3, scope.project.id=632d504031cb466299331db2d9184912, user.id=bd9193fd644f4ec7a39d9c6ab8a1e525) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:02.994613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9368-aea7-4dc2-8741-b011d033a74b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:02.996050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9368-aea7-4dc2-8741-b011d033a74b None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:03.001577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 643/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 22:53:02 2022] GET /identity/v3/role_assignments?user.id=bd9193fd644f4ec7a39d9c6ab8a1e525&scope.project.id=632d504031cb466299331db2d9184912&role.id=67db52226a7a4fd0b469e8de2c12b8e3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:05.127752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76488c-74f2-4da5-a8e6-cb3dc63ec53e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:05.128289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76488c-74f2-4da5-a8e6-cb3dc63ec53e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:05.128289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76488c-74f2-4da5-a8e6-cb3dc63ec53e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:05.129627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 645/1288] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:53:05.134742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b525908-5d54-482b-8476-b6df3c3558b4 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:05.135138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b525908-5d54-482b-8476-b6df3c3558b4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:05.135288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b525908-5d54-482b-8476-b6df3c3558b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:05.153521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9b525908-5d54-482b-8476-b6df3c3558b4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:53:05.154691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9b525908-5d54-482b-8476-b6df3c3558b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:53:05.183210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:05.220171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 644/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:53:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:53:05.501686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76488c-74f2-4da5-a8e6-cb3dc63ec53e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:05.599418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76488c-74f2-4da5-a8e6-cb3dc63ec53e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:05.609743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:05.634179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:05.634803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:05.635082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:05.635351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:05.635956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:05.637314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:05.645943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:05.647162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:05.648072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 646/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:53:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:05.653005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b525908-5d54-482b-8476-b6df3c3558b4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:05.750185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b525908-5d54-482b-8476-b6df3c3558b4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:05.783106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d9168-c8a4-437a-b527-62273ed7d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:05.783727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d9168-c8a4-437a-b527-62273ed7d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:05.784001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d9168-c8a4-437a-b527-62273ed7d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:05.784266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d9168-c8a4-437a-b527-62273ed7d1d4 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:05.784877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d9168-c8a4-437a-b527-62273ed7d1d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:05.785174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d9168-c8a4-437a-b527-62273ed7d1d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:05.786515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d9168-c8a4-437a-b527-62273ed7d1d4 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:05.791068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 645/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 22:53:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:05.797012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:05.894377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74228-7cf5-4903-b02c-024df3cf0a22 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:05.926569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c926e-350d-4ac3-bf46-a2fb9363767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:05.927150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c926e-350d-4ac3-bf46-a2fb9363767f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:05.927423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c926e-350d-4ac3-bf46-a2fb9363767f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:05.927718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c926e-350d-4ac3-bf46-a2fb9363767f None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:05.928330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c926e-350d-4ac3-bf46-a2fb9363767f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:05.928625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c926e-350d-4ac3-bf46-a2fb9363767f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:05.929967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c926e-350d-4ac3-bf46-a2fb9363767f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:05.934187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 647/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 22:53:05 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:08.042999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffee0-aa85-4a7a-bab3-87131d4acb64 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:08.043581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffee0-aa85-4a7a-bab3-87131d4acb64 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:08.043581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffee0-aa85-4a7a-bab3-87131d4acb64 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:08.044904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 646/1293] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:53:08.050761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b43dab-a612-4211-b72d-c85c379056fb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:08.051180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b43dab-a612-4211-b72d-c85c379056fb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:08.051286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b43dab-a612-4211-b72d-c85c379056fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:08.069790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-65b43dab-a612-4211-b72d-c85c379056fb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:53:08.070995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-65b43dab-a612-4211-b72d-c85c379056fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:53:08.099354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:08.136876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 648/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:53:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:53:08.411097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ffee0-aa85-4a7a-bab3-87131d4acb64 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:08.442525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:08.512836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ffee0-aa85-4a7a-bab3-87131d4acb64 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:08.546849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afe8d8-6579-4f53-b4aa-5fb4a83dd73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:08.547448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe8d8-6579-4f53-b4aa-5fb4a83dd73e None admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:08.548259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe8d8-6579-4f53-b4aa-5fb4a83dd73e None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:08.548524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afe8d8-6579-4f53-b4aa-5fb4a83dd73e None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:08.549169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afe8d8-6579-4f53-b4aa-5fb4a83dd73e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:08.550625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afe8d8-6579-4f53-b4aa-5fb4a83dd73e None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:08.565150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 647/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 22:53:08 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:53:10.748113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155be90-86a9-4e48-828d-9148dce0ccfd None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:10.748572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155be90-86a9-4e48-828d-9148dce0ccfd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:10.748683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155be90-86a9-4e48-828d-9148dce0ccfd None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:10.749977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 649/1296] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:53:10.755849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e5cf1-6a7d-42ec-b96f-ea790b3ddcab None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:10.756225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e5cf1-6a7d-42ec-b96f-ea790b3ddcab None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:10.756399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e5cf1-6a7d-42ec-b96f-ea790b3ddcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:10.774709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3d6e5cf1-6a7d-42ec-b96f-ea790b3ddcab None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:53:10.775926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3d6e5cf1-6a7d-42ec-b96f-ea790b3ddcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:53:10.845612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 648/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:53:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:53:11.111637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155be90-86a9-4e48-828d-9148dce0ccfd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:11.207803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155be90-86a9-4e48-828d-9148dce0ccfd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:11.238965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:11.239595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:11.239870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:11.240132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:11.240695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:11.242040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:11.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:11.252078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:11.253225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 650/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:53:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:11.257799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e5cf1-6a7d-42ec-b96f-ea790b3ddcab None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:11.279052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:11.370386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e5cf1-6a7d-42ec-b96f-ea790b3ddcab None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:11.403336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:11.403963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:11.404243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:11.404512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:11.405129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:11.405426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:11.406780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:11.411738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 649/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 22:53:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:11.417837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:11.448570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:11.513575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91852097-ad54-43ef-a42c-bbf633163b52 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:11.545756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a2ab-6910-4b3b-adc7-8ab2460045c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:11.546339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a2ab-6910-4b3b-adc7-8ab2460045c3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:11.546611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a2ab-6910-4b3b-adc7-8ab2460045c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:11.546875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a2ab-6910-4b3b-adc7-8ab2460045c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:11.548100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91a2ab-6910-4b3b-adc7-8ab2460045c3 None admin] RBAC: Adding query filter params (interface=public, service_id=2fcb2c28936545d28192013aa9de4cd5, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:11.548405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91a2ab-6910-4b3b-adc7-8ab2460045c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:11.549748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91a2ab-6910-4b3b-adc7-8ab2460045c3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:11.554512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 651/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 22:53:11 2022] GET /identity/v3/endpoints?service_id=2fcb2c28936545d28192013aa9de4cd5&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:11.560097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:11.639558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:11.657401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f91319-94b8-4faa-882f-76461bede8e5 None admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:11.690043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffa957-3b39-4f31-8ed4-b5a29789937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:11.690638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffa957-3b39-4f31-8ed4-b5a29789937d None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:11.690916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffa957-3b39-4f31-8ed4-b5a29789937d None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:11.691186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffa957-3b39-4f31-8ed4-b5a29789937d None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:11.691807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffa957-3b39-4f31-8ed4-b5a29789937d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:11.693151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffa957-3b39-4f31-8ed4-b5a29789937d None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:11.699665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 650/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 22:53:11 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 22:53:13.805284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0716-5753-48c0-bbee-8ff5d962b548 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:13.805775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0716-5753-48c0-bbee-8ff5d962b548 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:13.805867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0716-5753-48c0-bbee-8ff5d962b548 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:13.807150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 652/1302] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:53:13.812236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb9ee3-17f0-4165-9a5c-e7318db7b536 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:13.812658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb9ee3-17f0-4165-9a5c-e7318db7b536 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:13.812785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb9ee3-17f0-4165-9a5c-e7318db7b536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:13.831020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dbeb9ee3-17f0-4165-9a5c-e7318db7b536 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:53:13.832227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dbeb9ee3-17f0-4165-9a5c-e7318db7b536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:53:13.904495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 651/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:53:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 22:53:14.174345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd0716-5753-48c0-bbee-8ff5d962b548 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:14.268762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:14.275987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd0716-5753-48c0-bbee-8ff5d962b548 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:14.310544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:14.311174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:14.311457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:14.311761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:14.312342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:14.314208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 22:53:14.322796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 22:53:14.324007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 09 22:53:14.325039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 653/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 22:53:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 22:53:14.330541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb9ee3-17f0-4165-9a5c-e7318db7b536 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:14.403349 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:14.425038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb9ee3-17f0-4165-9a5c-e7318db7b536 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:14.457770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c86b2-1156-4aff-8cc7-6d9be15348ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:14.458358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c86b2-1156-4aff-8cc7-6d9be15348ae None admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:14.458633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c86b2-1156-4aff-8cc7-6d9be15348ae None admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:14.458899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c86b2-1156-4aff-8cc7-6d9be15348ae None admin] PATH_INFO: `/v3/services` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:14.459525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c86b2-1156-4aff-8cc7-6d9be15348ae None admin] RBAC: Adding query filter params (name=placement) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:53:14.459824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c86b2-1156-4aff-8cc7-6d9be15348ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:14.461157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c86b2-1156-4aff-8cc7-6d9be15348ae None admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:14.465981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 652/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 22:53:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:53:14.472153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:53:14.568286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc27ef-bf52-4f4c-96a1-baa06b266695 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:53:14.600876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c9dfd-2fcc-4ed6-ac42-664e7f78a7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:53:14.601456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9dfd-2fcc-4ed6-ac42-664e7f78a7d8 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:53:14.601732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9dfd-2fcc-4ed6-ac42-664e7f78a7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:53:14.601997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9dfd-2fcc-4ed6-ac42-664e7f78a7d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:53:14.602639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c9dfd-2fcc-4ed6-ac42-664e7f78a7d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:53:14.604014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c9dfd-2fcc-4ed6-ac42-664e7f78a7d8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:53:14.624514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 654/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:53:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:53:24.463144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=30 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:53:24.620324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:18.321003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571007-d793-4d56-b244-5321702ced61 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:18.322042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571007-d793-4d56-b244-5321702ced61 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:18.322533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571007-d793-4d56-b244-5321702ced61 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:18.324280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 653/1307] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:18.330892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275373d3-fe5b-436b-a4a5-d044d2c07755 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:18.331956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275373d3-fe5b-436b-a4a5-d044d2c07755 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:18.332477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275373d3-fe5b-436b-a4a5-d044d2c07755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:18.448150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-275373d3-fe5b-436b-a4a5-d044d2c07755 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:18.449523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-275373d3-fe5b-436b-a4a5-d044d2c07755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:18.569756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 655/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:18.880726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5fc91-dfa8-446d-af09-6848dff8da09 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:18.881072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5fc91-dfa8-446d-af09-6848dff8da09 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:18.881281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5fc91-dfa8-446d-af09-6848dff8da09 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:18.883001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 654/1309] 10.222.0.17 () {54 vars in 965 bytes} [Fri Sep 9 22:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:18.889698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683babb-6b70-4494-b603-adf197752cbf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:18.890043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683babb-6b70-4494-b603-adf197752cbf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:18.890254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683babb-6b70-4494-b603-adf197752cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:18.974133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5683babb-6b70-4494-b603-adf197752cbf None None] MFA Rules not processed for user `c0e10b1ef7454167bd1c606ef4f70bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:18.975478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5683babb-6b70-4494-b603-adf197752cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:18.999026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:19.063282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 656/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Fri Sep 9 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:19.072222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5fc91-dfa8-446d-af09-6848dff8da09 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:19.074045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0ee5fc91-dfa8-446d-af09-6848dff8da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:19.193759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5fc91-dfa8-446d-af09-6848dff8da09 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:19.283709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3602c39-21db-48af-90db-b7ff48182426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:19.284742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602c39-21db-48af-90db-b7ff48182426 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:19.285090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602c39-21db-48af-90db-b7ff48182426 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:19.285432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602c39-21db-48af-90db-b7ff48182426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:19.287235 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3602c39-21db-48af-90db-b7ff48182426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:19.351280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:19.415924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3602c39-21db-48af-90db-b7ff48182426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:19.417716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3602c39-21db-48af-90db-b7ff48182426 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:19.443141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 655/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:21.947120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586546c8-2b7c-419b-ab9a-d159a7c483bc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:21.947715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586546c8-2b7c-419b-ab9a-d159a7c483bc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:21.947715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586546c8-2b7c-419b-ab9a-d159a7c483bc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:21.948985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 657/1312] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:21.954374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dbe1f-c12a-42af-98f2-daf02a0a0ab5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:21.954776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dbe1f-c12a-42af-98f2-daf02a0a0ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:21.954923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dbe1f-c12a-42af-98f2-daf02a0a0ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:22.039566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e18dbe1f-c12a-42af-98f2-daf02a0a0ab5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:22.040783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e18dbe1f-c12a-42af-98f2-daf02a0a0ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:22.136315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 656/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:22.420976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586546c8-2b7c-419b-ab9a-d159a7c483bc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:22.422533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-586546c8-2b7c-419b-ab9a-d159a7c483bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:22.523005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586546c8-2b7c-419b-ab9a-d159a7c483bc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:22.593073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:22.595345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c04b0-3eaa-44f3-956f-9921a6ab2ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:22.596171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c04b0-3eaa-44f3-956f-9921a6ab2ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:22.596635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c04b0-3eaa-44f3-956f-9921a6ab2ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:22.597091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c04b0-3eaa-44f3-956f-9921a6ab2ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:22.598698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-799c04b0-3eaa-44f3-956f-9921a6ab2ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:22.705721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c04b0-3eaa-44f3-956f-9921a6ab2ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:22.707081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c04b0-3eaa-44f3-956f-9921a6ab2ad1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:22.725170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 658/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:25.201636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7c1c8-5943-4d7f-903b-0a36a86e736a None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:25.202182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7c1c8-5943-4d7f-903b-0a36a86e736a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:25.202182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7c1c8-5943-4d7f-903b-0a36a86e736a None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:25.203542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 657/1315] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:25.208969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531ca46-00e7-4eee-b82c-c2c80c0638cf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:25.209642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531ca46-00e7-4eee-b82c-c2c80c0638cf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:25.209642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531ca46-00e7-4eee-b82c-c2c80c0638cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:25.275335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2531ca46-00e7-4eee-b82c-c2c80c0638cf None None] MFA Rules not processed for user `8db4ab0dc8ec4234b9f8664feb8161e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:25.276619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2531ca46-00e7-4eee-b82c-c2c80c0638cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:25.328459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:25.372074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 659/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:54:25.660518 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7c1c8-5943-4d7f-903b-0a36a86e736a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:25.662082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-77b7c1c8-5943-4d7f-903b-0a36a86e736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:25.672864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:25.765585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7c1c8-5943-4d7f-903b-0a36a86e736a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:25.838282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ba8b2-f238-4aad-bcfe-cf8ad99c4122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:25.838885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ba8b2-f238-4aad-bcfe-cf8ad99c4122 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:25.839164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ba8b2-f238-4aad-bcfe-cf8ad99c4122 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:25.839431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ba8b2-f238-4aad-bcfe-cf8ad99c4122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:25.841058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-600ba8b2-f238-4aad-bcfe-cf8ad99c4122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:25.902202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:25.947951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ba8b2-f238-4aad-bcfe-cf8ad99c4122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:25.949313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ba8b2-f238-4aad-bcfe-cf8ad99c4122 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:25.967121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 658/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:28.622946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3ba7b-8860-4f3c-85a7-0cd4435af0eb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:28.623558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3ba7b-8860-4f3c-85a7-0cd4435af0eb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:28.623558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3ba7b-8860-4f3c-85a7-0cd4435af0eb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:28.624873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 660/1318] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:28.630008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc998-5883-47ad-a1cc-a8f7481d1643 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:28.630412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc998-5883-47ad-a1cc-a8f7481d1643 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:28.630559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc998-5883-47ad-a1cc-a8f7481d1643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:28.695694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-11ddc998-5883-47ad-a1cc-a8f7481d1643 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:28.696895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-11ddc998-5883-47ad-a1cc-a8f7481d1643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:28.791110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 659/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:29.079240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3ba7b-8860-4f3c-85a7-0cd4435af0eb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:29.080821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b4e3ba7b-8860-4f3c-85a7-0cd4435af0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:29.181245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3ba7b-8860-4f3c-85a7-0cd4435af0eb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:29.252665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e7835-14f4-462c-a51a-b1759c8fe3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:29.253265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7835-14f4-462c-a51a-b1759c8fe3ad service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:29.253544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7835-14f4-462c-a51a-b1759c8fe3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:29.253813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7835-14f4-462c-a51a-b1759c8fe3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:29.255440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-504e7835-14f4-462c-a51a-b1759c8fe3ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:29.262182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:29.365653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e7835-14f4-462c-a51a-b1759c8fe3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:29.367011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e7835-14f4-462c-a51a-b1759c8fe3ad service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:29.384407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 661/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:32.236503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaf141-a678-4175-b13f-359496573d80 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:32.237091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaf141-a678-4175-b13f-359496573d80 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:32.237091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaf141-a678-4175-b13f-359496573d80 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:32.238610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 660/1321] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:32.244172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20418c-8c7c-4d44-bade-84519be17cf9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:32.244716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20418c-8c7c-4d44-bade-84519be17cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:32.244837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20418c-8c7c-4d44-bade-84519be17cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:32.317552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4c20418c-8c7c-4d44-bade-84519be17cf9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:32.318690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4c20418c-8c7c-4d44-bade-84519be17cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:32.381977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:32.424122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 662/1322] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:32.704326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaf141-a678-4175-b13f-359496573d80 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:32.705867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-acbaf141-a678-4175-b13f-359496573d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:32.725967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:32.810534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaf141-a678-4175-b13f-359496573d80 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:32.882924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b8c5c-24e5-4c02-b43a-e931e9c280bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:32.883560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b8c5c-24e5-4c02-b43a-e931e9c280bd service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:32.883841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b8c5c-24e5-4c02-b43a-e931e9c280bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:32.884111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b8c5c-24e5-4c02-b43a-e931e9c280bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:32.885683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fe4b8c5c-24e5-4c02-b43a-e931e9c280bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:32.963524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:33.001860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b8c5c-24e5-4c02-b43a-e931e9c280bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:33.003296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b8c5c-24e5-4c02-b43a-e931e9c280bd service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:33.021428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 661/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:35.278145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed844333-8b45-4bd4-8469-d4de2e21e4f3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:35.278707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed844333-8b45-4bd4-8469-d4de2e21e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:35.278707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed844333-8b45-4bd4-8469-d4de2e21e4f3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:35.280859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 663/1324] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:35.286483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3f5dd-222c-4890-a0f7-51917debefe9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:35.286961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3f5dd-222c-4890-a0f7-51917debefe9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:35.287056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3f5dd-222c-4890-a0f7-51917debefe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:35.362332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4bd3f5dd-222c-4890-a0f7-51917debefe9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:35.363581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4bd3f5dd-222c-4890-a0f7-51917debefe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:35.462039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 662/1325] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:35.742735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed844333-8b45-4bd4-8469-d4de2e21e4f3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:35.744334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ed844333-8b45-4bd4-8469-d4de2e21e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:35.847154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed844333-8b45-4bd4-8469-d4de2e21e4f3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:35.922072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b81f62-54e6-4885-951e-64f501962253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:35.922677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81f62-54e6-4885-951e-64f501962253 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:35.922951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81f62-54e6-4885-951e-64f501962253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:35.923217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81f62-54e6-4885-951e-64f501962253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:35.924863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-22b81f62-54e6-4885-951e-64f501962253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:35.941761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:36.033528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b81f62-54e6-4885-951e-64f501962253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:36.034966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b81f62-54e6-4885-951e-64f501962253 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:36.053249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 664/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:38.252782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc68b2-853d-42ee-90cc-a67a9e705318 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:38.253294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc68b2-853d-42ee-90cc-a67a9e705318 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:38.253386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc68b2-853d-42ee-90cc-a67a9e705318 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:38.254719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 663/1327] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:38.259963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79889-c1cd-4cc1-8d2a-64ccb4dcbc54 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:38.260405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79889-c1cd-4cc1-8d2a-64ccb4dcbc54 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:38.260501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79889-c1cd-4cc1-8d2a-64ccb4dcbc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:38.324764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-60a79889-c1cd-4cc1-8d2a-64ccb4dcbc54 None None] MFA Rules not processed for user `8db4ab0dc8ec4234b9f8664feb8161e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:38.326579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60a79889-c1cd-4cc1-8d2a-64ccb4dcbc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:38.391400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:38.422988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 665/1328] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:38.706043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc68b2-853d-42ee-90cc-a67a9e705318 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:38.707603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bdfc68b2-853d-42ee-90cc-a67a9e705318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:38.739869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:38.813662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc68b2-853d-42ee-90cc-a67a9e705318 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:38.886297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ba09f-52e2-48ba-814e-48d0afcb0ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:38.886912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ba09f-52e2-48ba-814e-48d0afcb0ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:38.887187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ba09f-52e2-48ba-814e-48d0afcb0ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:38.887454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ba09f-52e2-48ba-814e-48d0afcb0ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:38.889610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f59ba09f-52e2-48ba-814e-48d0afcb0ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:38.965614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:38.996469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ba09f-52e2-48ba-814e-48d0afcb0ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:38.998374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ba09f-52e2-48ba-814e-48d0afcb0ef4 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:39.016635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 664/1329] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:41.431661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4bb4-58e2-4fb0-8a60-fb7dabec631a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:41.432219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4bb4-58e2-4fb0-8a60-fb7dabec631a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:41.432219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a4bb4-58e2-4fb0-8a60-fb7dabec631a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:41.433552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 666/1330] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:41.439337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa762-9f92-4057-b5ff-05000d5df810 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:41.439815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa762-9f92-4057-b5ff-05000d5df810 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:41.439932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa762-9f92-4057-b5ff-05000d5df810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:41.504933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ea2aa762-9f92-4057-b5ff-05000d5df810 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:41.506135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ea2aa762-9f92-4057-b5ff-05000d5df810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:41.603407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 665/1331] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:41.885395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a4bb4-58e2-4fb0-8a60-fb7dabec631a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:41.886927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b39a4bb4-58e2-4fb0-8a60-fb7dabec631a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:41.988947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a4bb4-58e2-4fb0-8a60-fb7dabec631a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:42.060307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d365c-23b2-46ee-af9a-1c47c27d8c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:42.060908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d365c-23b2-46ee-af9a-1c47c27d8c77 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:42.061187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d365c-23b2-46ee-af9a-1c47c27d8c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:42.061457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d365c-23b2-46ee-af9a-1c47c27d8c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:42.063038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e81d365c-23b2-46ee-af9a-1c47c27d8c77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:42.088785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:42.170094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d365c-23b2-46ee-af9a-1c47c27d8c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:42.171803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d365c-23b2-46ee-af9a-1c47c27d8c77 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:42.189830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 667/1332] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:44.386114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b6281-f14b-4c87-8d62-ab37747ceac3 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:44.386676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b6281-f14b-4c87-8d62-ab37747ceac3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:44.386676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b6281-f14b-4c87-8d62-ab37747ceac3 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:44.388075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 666/1333] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:44.393263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8f5b7-4ad9-4a03-97fc-5e25fe4b40e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:44.393674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8f5b7-4ad9-4a03-97fc-5e25fe4b40e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:44.393811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8f5b7-4ad9-4a03-97fc-5e25fe4b40e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:44.458986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4de8f5b7-4ad9-4a03-97fc-5e25fe4b40e9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:44.460222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4de8f5b7-4ad9-4a03-97fc-5e25fe4b40e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:44.534370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:44.556731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 668/1334] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:44.853102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dd494-82c3-4b34-82a3-9f42d538d3c1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:44.853385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dd494-82c3-4b34-82a3-9f42d538d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:44.853657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dd494-82c3-4b34-82a3-9f42d538d3c1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:44.854984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 667/1335] 10.222.0.17 () {54 vars in 965 bytes} [Fri Sep 9 22:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:44.861448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077a60b-8674-42e0-a7a9-6f58e40d49db None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:44.861732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077a60b-8674-42e0-a7a9-6f58e40d49db None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:44.862000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077a60b-8674-42e0-a7a9-6f58e40d49db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:44.937471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4077a60b-8674-42e0-a7a9-6f58e40d49db None None] MFA Rules not processed for user `c0e10b1ef7454167bd1c606ef4f70bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:44.938706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4077a60b-8674-42e0-a7a9-6f58e40d49db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:45.015015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 669/1336] 10.222.0.17 () {58 vars in 1062 bytes} [Fri Sep 9 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:54:45.023991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dd494-82c3-4b34-82a3-9f42d538d3c1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:45.025513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bb5dd494-82c3-4b34-82a3-9f42d538d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:45.065479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:45.128326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dd494-82c3-4b34-82a3-9f42d538d3c1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:45.200091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b2bcd-fed0-40c0-927b-92199f504aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:45.200775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b2bcd-fed0-40c0-927b-92199f504aab service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:45.201054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b2bcd-fed0-40c0-927b-92199f504aab service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:45.201324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b2bcd-fed0-40c0-927b-92199f504aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:45.202895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f32b2bcd-fed0-40c0-927b-92199f504aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:45.284938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:45.311781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b2bcd-fed0-40c0-927b-92199f504aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:45.313207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b2bcd-fed0-40c0-927b-92199f504aab service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:45.330697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 668/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:48.325294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19966837-e9e7-41e7-b58b-8addf433f827 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:48.325722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19966837-e9e7-41e7-b58b-8addf433f827 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:48.325852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19966837-e9e7-41e7-b58b-8addf433f827 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:48.327196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 670/1338] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:48.332754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fe7b3-940e-400d-84f8-ec26437bf27d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:48.333233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fe7b3-940e-400d-84f8-ec26437bf27d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:48.333326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fe7b3-940e-400d-84f8-ec26437bf27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:48.398267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3e0fe7b3-940e-400d-84f8-ec26437bf27d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:48.399479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3e0fe7b3-940e-400d-84f8-ec26437bf27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:48.494096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 669/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:54:48.780173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19966837-e9e7-41e7-b58b-8addf433f827 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:48.781702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-19966837-e9e7-41e7-b58b-8addf433f827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:48.858091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:48.885334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19966837-e9e7-41e7-b58b-8addf433f827 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:48.955929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e93445-21ea-420e-a831-1a13d48c2ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:48.956524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e93445-21ea-420e-a831-1a13d48c2ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:48.956797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e93445-21ea-420e-a831-1a13d48c2ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:48.957060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e93445-21ea-420e-a831-1a13d48c2ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:48.958628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-53e93445-21ea-420e-a831-1a13d48c2ee6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:49.071820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e93445-21ea-420e-a831-1a13d48c2ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:49.073258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e93445-21ea-420e-a831-1a13d48c2ee6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:49.090511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 671/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:53.295835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79823e1-9ed4-4f0a-b5f4-d3f00a591ba8 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:53.296420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79823e1-9ed4-4f0a-b5f4-d3f00a591ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:53.296420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79823e1-9ed4-4f0a-b5f4-d3f00a591ba8 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:53.297706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 670/1341] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:54:53.302884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046838-964a-4e47-8ee1-9d18f0343c64 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:53.303290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046838-964a-4e47-8ee1-9d18f0343c64 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:53.303434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046838-964a-4e47-8ee1-9d18f0343c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:53.309140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:53.369635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8f046838-964a-4e47-8ee1-9d18f0343c64 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:53.370741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f046838-964a-4e47-8ee1-9d18f0343c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:53.464420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 672/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:53.747052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79823e1-9ed4-4f0a-b5f4-d3f00a591ba8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:53.748647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e79823e1-9ed4-4f0a-b5f4-d3f00a591ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:53.799404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:53.854548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79823e1-9ed4-4f0a-b5f4-d3f00a591ba8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:53.926467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1481-bd12-417d-8241-8e120b61c06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:53.927069 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1481-bd12-417d-8241-8e120b61c06c service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:53.927343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1481-bd12-417d-8241-8e120b61c06c service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:53.927642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1481-bd12-417d-8241-8e120b61c06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:53.929208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-08cb1481-bd12-417d-8241-8e120b61c06c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:54.020647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:54.037165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb1481-bd12-417d-8241-8e120b61c06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:54.038588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb1481-bd12-417d-8241-8e120b61c06c service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:54.057230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 671/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:54:56.826542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14976a-4406-4ad5-8150-c8ca880d048b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:56.827356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14976a-4406-4ad5-8150-c8ca880d048b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:56.827577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14976a-4406-4ad5-8150-c8ca880d048b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:56.829009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 673/1344] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:54:56.834224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd7f78-3b49-4acf-8ef2-ac258d149308 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:56.834648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd7f78-3b49-4acf-8ef2-ac258d149308 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:56.834759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd7f78-3b49-4acf-8ef2-ac258d149308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:56.899945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-88cd7f78-3b49-4acf-8ef2-ac258d149308 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:54:56.901155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-88cd7f78-3b49-4acf-8ef2-ac258d149308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:56.996453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 672/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:54:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:54:57.277214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14976a-4406-4ad5-8150-c8ca880d048b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:54:57.279379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ba14976a-4406-4ad5-8150-c8ca880d048b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:57.369815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:54:57.387234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14976a-4406-4ad5-8150-c8ca880d048b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:54:57.468017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540111f-f7a7-4ca2-ad2b-68fa2f5461a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:54:57.468642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540111f-f7a7-4ca2-ad2b-68fa2f5461a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:54:57.468942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540111f-f7a7-4ca2-ad2b-68fa2f5461a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:54:57.469212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540111f-f7a7-4ca2-ad2b-68fa2f5461a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:54:57.470920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e540111f-f7a7-4ca2-ad2b-68fa2f5461a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:54:57.579607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540111f-f7a7-4ca2-ad2b-68fa2f5461a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:54:57.580968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540111f-f7a7-4ca2-ad2b-68fa2f5461a6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:54:57.599815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 674/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:54:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:01.999386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdb207-0895-42a8-9a2a-5e8682b8c5c2 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:01.999874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdb207-0895-42a8-9a2a-5e8682b8c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:02.000047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdb207-0895-42a8-9a2a-5e8682b8c5c2 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:02.001563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 673/1347] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:02.007151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88915bc3-b4e1-4fea-996a-3ce46a5c90d0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:02.007706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88915bc3-b4e1-4fea-996a-3ce46a5c90d0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:02.007837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88915bc3-b4e1-4fea-996a-3ce46a5c90d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:02.023855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:02.082246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-88915bc3-b4e1-4fea-996a-3ce46a5c90d0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:02.083652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-88915bc3-b4e1-4fea-996a-3ce46a5c90d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:02.199625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 675/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:02.488875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdb207-0895-42a8-9a2a-5e8682b8c5c2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:02.490432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7ecdb207-0895-42a8-9a2a-5e8682b8c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:02.549119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:02.594680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdb207-0895-42a8-9a2a-5e8682b8c5c2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:02.665488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8b4bd-b689-40cf-a728-b209b277c2da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:02.666086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b4bd-b689-40cf-a728-b209b277c2da service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:02.666365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b4bd-b689-40cf-a728-b209b277c2da service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:02.666634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b4bd-b689-40cf-a728-b209b277c2da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:02.668245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-89b8b4bd-b689-40cf-a728-b209b277c2da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:02.772044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:02.776638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8b4bd-b689-40cf-a728-b209b277c2da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:02.778691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8b4bd-b689-40cf-a728-b209b277c2da service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:02.796584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 674/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:07.696504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03923fc9-69a4-489c-837b-9aef7efbe632 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:07.697698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03923fc9-69a4-489c-837b-9aef7efbe632 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:07.698748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03923fc9-69a4-489c-837b-9aef7efbe632 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:07.701014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 676/1350] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 22:55:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:07.706064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b580fd-3ba3-46af-8672-8b45b131b45d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:07.706531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b580fd-3ba3-46af-8672-8b45b131b45d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:07.706616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b580fd-3ba3-46af-8672-8b45b131b45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:07.824259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-58b580fd-3ba3-46af-8672-8b45b131b45d None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:07.826815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-58b580fd-3ba3-46af-8672-8b45b131b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:07.988701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 675/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 22:55:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:12.148323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aa2e6-8915-4c8b-b252-af858a32f817 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:12.148885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aa2e6-8915-4c8b-b252-af858a32f817 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:12.148885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aa2e6-8915-4c8b-b252-af858a32f817 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:12.150295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 677/1352] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 22:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:12.156012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab7c7e-6ffa-467d-b263-7c758bfcf2d8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:12.156463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab7c7e-6ffa-467d-b263-7c758bfcf2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:12.156691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab7c7e-6ffa-467d-b263-7c758bfcf2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:12.196361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=59 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:12.215019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:12.240928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-83ab7c7e-6ffa-467d-b263-7c758bfcf2d8 None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:12.242179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-83ab7c7e-6ffa-467d-b263-7c758bfcf2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:12.353749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 676/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 22:55:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:13.796772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371c52-ea44-4272-a2e5-50a8ecf9aeff None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:13.797295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371c52-ea44-4272-a2e5-50a8ecf9aeff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:13.797295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371c52-ea44-4272-a2e5-50a8ecf9aeff None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:13.798670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 678/1354] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 22:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:13.803636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80739d07-1d07-4eef-82ed-99f9908d2690 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:13.804082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80739d07-1d07-4eef-82ed-99f9908d2690 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:13.804182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80739d07-1d07-4eef-82ed-99f9908d2690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:13.884396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-80739d07-1d07-4eef-82ed-99f9908d2690 None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:13.885806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-80739d07-1d07-4eef-82ed-99f9908d2690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:13.932657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:14.012828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 677/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 22:55:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:55:17.398626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c50b1-c09c-4612-b13c-4036fc9b9083 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:17.399154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c50b1-c09c-4612-b13c-4036fc9b9083 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:17.399154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c50b1-c09c-4612-b13c-4036fc9b9083 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:17.400481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 679/1356] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:17.406550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b694a-ad01-4cd7-a94a-d60319a18d20 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:17.406984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b694a-ad01-4cd7-a94a-d60319a18d20 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:17.407108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b694a-ad01-4cd7-a94a-d60319a18d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:17.482465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3b3b694a-ad01-4cd7-a94a-d60319a18d20 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:17.483767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3b3b694a-ad01-4cd7-a94a-d60319a18d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:17.584890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 678/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:17.899791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d6c50-d87a-4801-9c07-4f08e872ea98 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:17.900173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d6c50-d87a-4801-9c07-4f08e872ea98 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:17.900715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d6c50-d87a-4801-9c07-4f08e872ea98 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:17.902144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 680/1358] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 22:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:17.910593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500f5d9-abc3-4763-80fc-cc5febba3a80 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:17.910886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500f5d9-abc3-4763-80fc-cc5febba3a80 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:17.911168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500f5d9-abc3-4763-80fc-cc5febba3a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:17.935853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:18.013195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6500f5d9-abc3-4763-80fc-cc5febba3a80 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:18.014425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6500f5d9-abc3-4763-80fc-cc5febba3a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:18.138266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 679/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 22:55:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:18.146128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d6c50-d87a-4801-9c07-4f08e872ea98 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:18.147889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e19d6c50-d87a-4801-9c07-4f08e872ea98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:18.312971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d6c50-d87a-4801-9c07-4f08e872ea98 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:18.399832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:18.444378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb1dd0-d137-489f-9583-07a8862b8f3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:18.445000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb1dd0-d137-489f-9583-07a8862b8f3d service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:18.445277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb1dd0-d137-489f-9583-07a8862b8f3d service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:18.445549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb1dd0-d137-489f-9583-07a8862b8f3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:18.447195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4ccb1dd0-d137-489f-9583-07a8862b8f3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:18.549319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb1dd0-d137-489f-9583-07a8862b8f3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:18.550922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb1dd0-d137-489f-9583-07a8862b8f3d service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:18.570354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 681/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:20.953736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df902f5-9f0f-41d2-9a8f-833703a5828e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:20.954368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df902f5-9f0f-41d2-9a8f-833703a5828e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:20.954368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df902f5-9f0f-41d2-9a8f-833703a5828e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:20.959364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 680/1361] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:20.973394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc402f-6480-48a7-8ada-202118b5cc2f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:20.973394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc402f-6480-48a7-8ada-202118b5cc2f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:20.973394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc402f-6480-48a7-8ada-202118b5cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:21.041724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-09fc402f-6480-48a7-8ada-202118b5cc2f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:21.043555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-09fc402f-6480-48a7-8ada-202118b5cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:21.084746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:21.154136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 682/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:21.448519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df902f5-9f0f-41d2-9a8f-833703a5828e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:21.450125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4df902f5-9f0f-41d2-9a8f-833703a5828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:21.522984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:21.628345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df902f5-9f0f-41d2-9a8f-833703a5828e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:21.759343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04463db9-3675-4b75-871d-4d236de30bb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:21.759981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04463db9-3675-4b75-871d-4d236de30bb2 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:21.760256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04463db9-3675-4b75-871d-4d236de30bb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:21.760524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04463db9-3675-4b75-871d-4d236de30bb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:21.762130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-04463db9-3675-4b75-871d-4d236de30bb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:21.785974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:21.860071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04463db9-3675-4b75-871d-4d236de30bb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:21.861603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04463db9-3675-4b75-871d-4d236de30bb2 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:21.879966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 681/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:24.373976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976a8f-429f-4c09-847e-24bbfc68d972 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:24.374519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976a8f-429f-4c09-847e-24bbfc68d972 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:24.374519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976a8f-429f-4c09-847e-24bbfc68d972 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:24.375898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 683/1364] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:24.381245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c6d22-938e-4a9a-a834-cbc6972a3eeb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:24.381673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c6d22-938e-4a9a-a834-cbc6972a3eeb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:24.381801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c6d22-938e-4a9a-a834-cbc6972a3eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:24.448860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9c5c6d22-938e-4a9a-a834-cbc6972a3eeb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:24.450608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9c5c6d22-938e-4a9a-a834-cbc6972a3eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:24.550714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:24.553368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 682/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:24.846049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac20-4d94-4666-9ac1-66d97907c068 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:24.846532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac20-4d94-4666-9ac1-66d97907c068 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:24.847178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac20-4d94-4666-9ac1-66d97907c068 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:24.848795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 684/1366] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 22:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:24.854125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201eb9c-b2e1-4cf7-b8ee-38c8a5104908 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:24.854596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201eb9c-b2e1-4cf7-b8ee-38c8a5104908 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:24.855040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201eb9c-b2e1-4cf7-b8ee-38c8a5104908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:24.934951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f201eb9c-b2e1-4cf7-b8ee-38c8a5104908 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:24.936460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f201eb9c-b2e1-4cf7-b8ee-38c8a5104908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:25.063801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 683/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 22:55:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:25.067900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9ac20-4d94-4666-9ac1-66d97907c068 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:25.069720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dac9ac20-4d94-4666-9ac1-66d97907c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:25.224084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9ac20-4d94-4666-9ac1-66d97907c068 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:25.245570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:25.347369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a758f-35aa-4c0f-8813-3e67f1055992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:25.348161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a758f-35aa-4c0f-8813-3e67f1055992 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:25.348532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a758f-35aa-4c0f-8813-3e67f1055992 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:25.348894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a758f-35aa-4c0f-8813-3e67f1055992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:25.350566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-682a758f-35aa-4c0f-8813-3e67f1055992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:25.447413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a758f-35aa-4c0f-8813-3e67f1055992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:25.449358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a758f-35aa-4c0f-8813-3e67f1055992 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:25.472237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 685/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:25.579672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204f86-ca3d-49c5-ab50-6308b9ae73d8 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:25.579672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204f86-ca3d-49c5-ab50-6308b9ae73d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:25.579672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204f86-ca3d-49c5-ab50-6308b9ae73d8 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:25.579672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 684/1369] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 22:55:25.581389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3247bc7-17cc-4943-9d5f-bfe5cb26a6f6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:25.581679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3247bc7-17cc-4943-9d5f-bfe5cb26a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:25.581953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3247bc7-17cc-4943-9d5f-bfe5cb26a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:25.626746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:25.665073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a3247bc7-17cc-4943-9d5f-bfe5cb26a6f6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:25.666512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a3247bc7-17cc-4943-9d5f-bfe5cb26a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:25.795761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 686/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:26.128346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fbe4b-bd36-43b6-a0e1-17046170db5b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:26.128346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fbe4b-bd36-43b6-a0e1-17046170db5b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:26.128346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fbe4b-bd36-43b6-a0e1-17046170db5b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:26.130517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 685/1371] 10.222.0.17 () {54 vars in 964 bytes} [Fri Sep 9 22:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:26.134751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e79bbc-c7b4-4ef4-8d7a-7737e3c47daf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:26.135051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e79bbc-c7b4-4ef4-8d7a-7737e3c47daf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:26.135323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e79bbc-c7b4-4ef4-8d7a-7737e3c47daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:26.215379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c3e79bbc-c7b4-4ef4-8d7a-7737e3c47daf None None] MFA Rules not processed for user `56228e09c8b742b29e49c40d1a39870f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:26.216616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c3e79bbc-c7b4-4ef4-8d7a-7737e3c47daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:26.268299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:26.303080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 687/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Sep 9 22:55:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:26.311260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fbe4b-bd36-43b6-a0e1-17046170db5b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:26.312879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c42fbe4b-bd36-43b6-a0e1-17046170db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:26.366869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:26.424464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fbe4b-bd36-43b6-a0e1-17046170db5b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:26.510006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe13dc-a4fc-41e4-b616-66d4006d9355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:26.510621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe13dc-a4fc-41e4-b616-66d4006d9355 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:26.510904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe13dc-a4fc-41e4-b616-66d4006d9355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:26.511181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe13dc-a4fc-41e4-b616-66d4006d9355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:26.513395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c9fe13dc-a4fc-41e4-b616-66d4006d9355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:26.630515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:26.646740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe13dc-a4fc-41e4-b616-66d4006d9355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:26.648851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe13dc-a4fc-41e4-b616-66d4006d9355 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:26.674073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 686/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:27.724237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb95ad-029c-468f-a03c-1b71c2888f11 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:27.724754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb95ad-029c-468f-a03c-1b71c2888f11 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:27.724858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb95ad-029c-468f-a03c-1b71c2888f11 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:27.726259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 688/1374] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:27.733863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b964f-7f97-4fe8-ac21-e211f7ce438b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:27.734254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b964f-7f97-4fe8-ac21-e211f7ce438b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:27.734561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b964f-7f97-4fe8-ac21-e211f7ce438b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:27.799930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a23b964f-7f97-4fe8-ac21-e211f7ce438b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:27.801257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a23b964f-7f97-4fe8-ac21-e211f7ce438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:27.901303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 687/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:28.204024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb95ad-029c-468f-a03c-1b71c2888f11 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:28.205630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-87eb95ad-029c-468f-a03c-1b71c2888f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:28.359128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb95ad-029c-468f-a03c-1b71c2888f11 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:28.412607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:28.475754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dae57-5b48-4002-8473-9203b878d4b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:28.476362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dae57-5b48-4002-8473-9203b878d4b7 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:28.476641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dae57-5b48-4002-8473-9203b878d4b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:28.476907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dae57-5b48-4002-8473-9203b878d4b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:28.478491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c22dae57-5b48-4002-8473-9203b878d4b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:28.571630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dae57-5b48-4002-8473-9203b878d4b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:28.573228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dae57-5b48-4002-8473-9203b878d4b7 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:28.591707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 689/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:30.856487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628e692-e935-40ac-8a41-aac215a0fabb None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:30.857022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628e692-e935-40ac-8a41-aac215a0fabb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:30.857144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628e692-e935-40ac-8a41-aac215a0fabb None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:30.858578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 688/1377] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:30.863727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32f7d-c814-4c17-8deb-86f0dd88cafe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:30.864169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32f7d-c814-4c17-8deb-86f0dd88cafe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:30.864338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32f7d-c814-4c17-8deb-86f0dd88cafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:30.939792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e1c32f7d-c814-4c17-8deb-86f0dd88cafe None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:30.940989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1c32f7d-c814-4c17-8deb-86f0dd88cafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:30.948583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:31.044508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 690/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:55:31.333164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628e692-e935-40ac-8a41-aac215a0fabb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:31.334664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9628e692-e935-40ac-8a41-aac215a0fabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:31.397850 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:31.484780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628e692-e935-40ac-8a41-aac215a0fabb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:31.598601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c87e4-cb32-4544-a1f5-fac94c0e0796 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:31.599122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c87e4-cb32-4544-a1f5-fac94c0e0796 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:31.599397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c87e4-cb32-4544-a1f5-fac94c0e0796 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:31.599688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c87e4-cb32-4544-a1f5-fac94c0e0796 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:31.601260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-201c87e4-cb32-4544-a1f5-fac94c0e0796 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:31.619516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:31.695044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c87e4-cb32-4544-a1f5-fac94c0e0796 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:31.696678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c87e4-cb32-4544-a1f5-fac94c0e0796 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:31.719032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 689/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:33.981497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9fd8-d8f9-4c73-b779-adb0ae747228 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:33.981950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9fd8-d8f9-4c73-b779-adb0ae747228 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:33.982668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9fd8-d8f9-4c73-b779-adb0ae747228 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:33.989329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 691/1380] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:33.990849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bcf7a-ef2d-4baf-a88a-a40c5bcce6fd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:33.990849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bcf7a-ef2d-4baf-a88a-a40c5bcce6fd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:33.990849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bcf7a-ef2d-4baf-a88a-a40c5bcce6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:34.084740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-100bcf7a-ef2d-4baf-a88a-a40c5bcce6fd None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:34.086695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-100bcf7a-ef2d-4baf-a88a-a40c5bcce6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:34.189911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:34.196374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 690/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:55:34.494651 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe9fd8-d8f9-4c73-b779-adb0ae747228 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:34.496761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5bfe9fd8-d8f9-4c73-b779-adb0ae747228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:34.647269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:34.658796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe9fd8-d8f9-4c73-b779-adb0ae747228 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:34.789003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8abf82-a87e-4527-a21b-576cc9863bd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:34.789625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8abf82-a87e-4527-a21b-576cc9863bd6 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:34.789904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8abf82-a87e-4527-a21b-576cc9863bd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:34.790173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8abf82-a87e-4527-a21b-576cc9863bd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:34.791805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ce8abf82-a87e-4527-a21b-576cc9863bd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:34.883244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8abf82-a87e-4527-a21b-576cc9863bd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:34.885037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8abf82-a87e-4527-a21b-576cc9863bd6 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:34.895692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:34.905188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 692/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:37.142430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18bdff-ee75-4673-9962-c8b8d56ae2e4 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:37.142990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18bdff-ee75-4673-9962-c8b8d56ae2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:37.142990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18bdff-ee75-4673-9962-c8b8d56ae2e4 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:37.144595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 691/1383] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:37.150023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f97b-3982-4d09-9cc2-9214c874816e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:37.151052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f97b-3982-4d09-9cc2-9214c874816e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:37.151052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f97b-3982-4d09-9cc2-9214c874816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:37.223461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-97c5f97b-3982-4d09-9cc2-9214c874816e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:37.224720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-97c5f97b-3982-4d09-9cc2-9214c874816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:37.338033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 693/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:37.634504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18bdff-ee75-4673-9962-c8b8d56ae2e4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:37.636038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ae18bdff-ee75-4673-9962-c8b8d56ae2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:37.800217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18bdff-ee75-4673-9962-c8b8d56ae2e4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:37.869722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:37.925214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5fded-9bde-4928-9b2c-fc67c6d18e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:37.926167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5fded-9bde-4928-9b2c-fc67c6d18e82 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:37.926532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5fded-9bde-4928-9b2c-fc67c6d18e82 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:37.926874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5fded-9bde-4928-9b2c-fc67c6d18e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:37.928963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aac5fded-9bde-4928-9b2c-fc67c6d18e82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:38.030332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5fded-9bde-4928-9b2c-fc67c6d18e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:38.032016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5fded-9bde-4928-9b2c-fc67c6d18e82 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:38.051659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 692/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:40.180096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12e26-0ada-4a3b-aa9f-9a2b8ec8780f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:40.180641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12e26-0ada-4a3b-aa9f-9a2b8ec8780f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:40.180641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12e26-0ada-4a3b-aa9f-9a2b8ec8780f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:40.182034 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 694/1386] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:40.187429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69696abf-72e7-43c4-8bfd-bdaf3e57fc04 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:40.187869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69696abf-72e7-43c4-8bfd-bdaf3e57fc04 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:40.188015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69696abf-72e7-43c4-8bfd-bdaf3e57fc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:40.254106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-69696abf-72e7-43c4-8bfd-bdaf3e57fc04 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:40.255362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-69696abf-72e7-43c4-8bfd-bdaf3e57fc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:40.289282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:40.354609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 693/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:40.640100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc12e26-0ada-4a3b-aa9f-9a2b8ec8780f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:40.641622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9fc12e26-0ada-4a3b-aa9f-9a2b8ec8780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:40.733957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:40.802409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc12e26-0ada-4a3b-aa9f-9a2b8ec8780f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:40.915621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c17b40-8e86-442e-b2a8-b08d0442bc4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:40.916149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c17b40-8e86-442e-b2a8-b08d0442bc4e service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:40.916431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c17b40-8e86-442e-b2a8-b08d0442bc4e service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:40.916703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c17b40-8e86-442e-b2a8-b08d0442bc4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:40.918826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d4c17b40-8e86-442e-b2a8-b08d0442bc4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:40.960715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:41.012751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c17b40-8e86-442e-b2a8-b08d0442bc4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:41.014425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c17b40-8e86-442e-b2a8-b08d0442bc4e service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:41.032631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 695/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:42.327921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c99c12-639e-4c86-9b1b-9f5b7dc28aa5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:42.328505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c99c12-639e-4c86-9b1b-9f5b7dc28aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:42.328505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c99c12-639e-4c86-9b1b-9f5b7dc28aa5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:42.331646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 694/1389] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:42.337123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792b20-b311-4604-b79b-2c9448d70abe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:42.337564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792b20-b311-4604-b79b-2c9448d70abe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:42.337669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792b20-b311-4604-b79b-2c9448d70abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:42.405535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ef792b20-b311-4604-b79b-2c9448d70abe None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:42.406759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ef792b20-b311-4604-b79b-2c9448d70abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:42.503656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 696/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:55:42.832397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd94c2-deb0-464a-be6b-e6b810f075ec None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:42.832776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd94c2-deb0-464a-be6b-e6b810f075ec None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:42.833058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd94c2-deb0-464a-be6b-e6b810f075ec None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:42.834474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 695/1391] 10.222.0.17 () {54 vars in 966 bytes} [Fri Sep 9 22:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:42.840817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07cd2-cdbc-47a2-bc57-ea416483c7c3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:42.841102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07cd2-cdbc-47a2-bc57-ea416483c7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:42.841374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07cd2-cdbc-47a2-bc57-ea416483c7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:42.858534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:42.925296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d0f07cd2-cdbc-47a2-bc57-ea416483c7c3 None None] MFA Rules not processed for user `6cf808a1f5dd45bea0a3a81131532b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:42.926608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d0f07cd2-cdbc-47a2-bc57-ea416483c7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:43.012818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 697/1392] 10.222.0.17 () {58 vars in 1063 bytes} [Fri Sep 9 22:55:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:43.022230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd94c2-deb0-464a-be6b-e6b810f075ec None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:43.023787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0bcd94c2-deb0-464a-be6b-e6b810f075ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:43.130949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd94c2-deb0-464a-be6b-e6b810f075ec None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:43.191152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:43.207004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:43.207642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:43.207921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:43.208189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:43.209754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:43.226397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fd4b-0185-4201-b747-6ec9fc22570b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:43.226916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fd4b-0185-4201-b747-6ec9fc22570b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:43.227400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fd4b-0185-4201-b747-6ec9fc22570b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:43.230282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 698/1393] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:43.235589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e655e-2e72-41c9-b2e0-8fa019368d39 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:43.235874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e655e-2e72-41c9-b2e0-8fa019368d39 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:43.236158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e655e-2e72-41c9-b2e0-8fa019368d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:43.303148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5e4e655e-2e72-41c9-b2e0-8fa019368d39 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:43.304314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5e4e655e-2e72-41c9-b2e0-8fa019368d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:43.334623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:43.336115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:43.355402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 696/1394] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:43.359037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:43.401217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 699/1395] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:55:43.698144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:43.699433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:43.806444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:43.814994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba145-fede-4743-aadd-39b37fed642d service glance] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:43.931740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5e607-1a72-4764-8180-77259f1ea681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:43.932363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5e607-1a72-4764-8180-77259f1ea681 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:43.932640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5e607-1a72-4764-8180-77259f1ea681 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:43.932906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5e607-1a72-4764-8180-77259f1ea681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:43.934605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-27a5e607-1a72-4764-8180-77259f1ea681 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:44.033685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5e607-1a72-4764-8180-77259f1ea681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:44.035283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5e607-1a72-4764-8180-77259f1ea681 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:44.050899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:44.054512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 697/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:44.116462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861cc3f-d0f5-4648-b0e9-d3d576bccd9a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:44.116795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861cc3f-d0f5-4648-b0e9-d3d576bccd9a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:44.117191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861cc3f-d0f5-4648-b0e9-d3d576bccd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:44.205331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f861cc3f-d0f5-4648-b0e9-d3d576bccd9a None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:44.206782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f861cc3f-d0f5-4648-b0e9-d3d576bccd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:44.313640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 700/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 22:55:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:44.362033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4ab7-0d84-4230-96a9-875a2d3df3b1 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:44.362557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4ab7-0d84-4230-96a9-875a2d3df3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:44.362675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4ab7-0d84-4230-96a9-875a2d3df3b1 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:44.364345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 698/1398] 10.222.0.17 () {54 vars in 943 bytes} [Fri Sep 9 22:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:44.371375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5d9f-8780-4f80-90bd-4bc97deeaaef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:44.371824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5d9f-8780-4f80-90bd-4bc97deeaaef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:44.372175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5d9f-8780-4f80-90bd-4bc97deeaaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:44.438243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:44.460167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e28d5d9f-8780-4f80-90bd-4bc97deeaaef None None] MFA Rules not processed for user `e7ceb38886d34f9d8e59b2871989446c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:44.461550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e28d5d9f-8780-4f80-90bd-4bc97deeaaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:44.582987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 701/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Fri Sep 9 22:55:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:44.593336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4ab7-0d84-4230-96a9-875a2d3df3b1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:44.595099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d3ef4ab7-0d84-4230-96a9-875a2d3df3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:44.736358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4ab7-0d84-4230-96a9-875a2d3df3b1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:44.821749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:44.847640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5785c-aba5-40f8-b806-1fc105c1891e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:44.848461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5785c-aba5-40f8-b806-1fc105c1891e service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:44.848991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5785c-aba5-40f8-b806-1fc105c1891e service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:44.849490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5785c-aba5-40f8-b806-1fc105c1891e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:44.852040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-91b5785c-aba5-40f8-b806-1fc105c1891e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:44.938111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5785c-aba5-40f8-b806-1fc105c1891e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:44.939997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5785c-aba5-40f8-b806-1fc105c1891e service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:44.950080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 699/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 22:55:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 22:55:46.250665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976084d-2aff-4bb3-a9ab-62e773503d5e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:46.251259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976084d-2aff-4bb3-a9ab-62e773503d5e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:46.251259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976084d-2aff-4bb3-a9ab-62e773503d5e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:46.252733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 702/1401] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:46.257885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9043672-053b-4df1-92a2-f08cf0be49e0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:46.258391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9043672-053b-4df1-92a2-f08cf0be49e0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:46.258487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9043672-053b-4df1-92a2-f08cf0be49e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:46.330421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a9043672-053b-4df1-92a2-f08cf0be49e0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:46.331672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a9043672-053b-4df1-92a2-f08cf0be49e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:46.407599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:46.441172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 700/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:46.786926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976084d-2aff-4bb3-a9ab-62e773503d5e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:46.788567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9976084d-2aff-4bb3-a9ab-62e773503d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:46.906908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:46.953843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976084d-2aff-4bb3-a9ab-62e773503d5e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:47.074804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581af02-df76-432e-b2de-e20b545265fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:47.075453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581af02-df76-432e-b2de-e20b545265fb service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:47.075761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581af02-df76-432e-b2de-e20b545265fb service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:47.076029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581af02-df76-432e-b2de-e20b545265fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:47.077705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b581af02-df76-432e-b2de-e20b545265fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:47.155279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:47.177515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581af02-df76-432e-b2de-e20b545265fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:47.179130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581af02-df76-432e-b2de-e20b545265fb service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:47.198204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 703/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:49.453635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7624534-89ed-4721-9d1c-121961828c56 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:49.454262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7624534-89ed-4721-9d1c-121961828c56 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:49.454262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7624534-89ed-4721-9d1c-121961828c56 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:49.455696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 701/1404] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:49.460912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc95445-a153-42b2-b398-e2838075c2de None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:49.461392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc95445-a153-42b2-b398-e2838075c2de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:49.461508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc95445-a153-42b2-b398-e2838075c2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:49.533369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1cc95445-a153-42b2-b398-e2838075c2de None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:49.534621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1cc95445-a153-42b2-b398-e2838075c2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:49.651836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 704/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:49.943977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7624534-89ed-4721-9d1c-121961828c56 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:49.945517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a7624534-89ed-4721-9d1c-121961828c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:50.113247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7624534-89ed-4721-9d1c-121961828c56 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:50.200383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:50.311271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9bb6d-a51e-4193-8361-f9f9ad413031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:50.311924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9bb6d-a51e-4193-8361-f9f9ad413031 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:50.312205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9bb6d-a51e-4193-8361-f9f9ad413031 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:50.312473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9bb6d-a51e-4193-8361-f9f9ad413031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:50.314138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dbf9bb6d-a51e-4193-8361-f9f9ad413031 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:50.419207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9bb6d-a51e-4193-8361-f9f9ad413031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:50.421172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9bb6d-a51e-4193-8361-f9f9ad413031 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:50.440619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 702/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:55:52.664766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3db2d-8a17-4b66-a69d-2ce9b835f0c1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:52.665344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3db2d-8a17-4b66-a69d-2ce9b835f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:52.665344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3db2d-8a17-4b66-a69d-2ce9b835f0c1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:52.666686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 705/1407] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:55:52.672847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6505a-6ef0-4bd0-b8cd-3db74a50137a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:52.673267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6505a-6ef0-4bd0-b8cd-3db74a50137a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:52.673453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6505a-6ef0-4bd0-b8cd-3db74a50137a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:52.748918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-55f6505a-6ef0-4bd0-b8cd-3db74a50137a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:55:52.752794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-55f6505a-6ef0-4bd0-b8cd-3db74a50137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:52.765482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:52.857795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 703/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:55:53.143731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3db2d-8a17-4b66-a69d-2ce9b835f0c1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:55:53.145259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-baa3db2d-8a17-4b66-a69d-2ce9b835f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:53.200638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:53.309408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3db2d-8a17-4b66-a69d-2ce9b835f0c1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:55:53.433067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb0c66-4a5a-4630-b28f-25929bb8db42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:55:53.433450 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb0c66-4a5a-4630-b28f-25929bb8db42 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:55:53.433848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb0c66-4a5a-4630-b28f-25929bb8db42 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:55:53.434193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb0c66-4a5a-4630-b28f-25929bb8db42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:55:53.435994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eebb0c66-4a5a-4630-b28f-25929bb8db42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:55:53.447416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:55:53.535629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb0c66-4a5a-4630-b28f-25929bb8db42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:55:53.537656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb0c66-4a5a-4630-b28f-25929bb8db42 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:55:53.560082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 706/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:02.855881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=37 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:03.555413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=41 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:07.869105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980c341-6f40-4911-8078-925f5f0f6962 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:07.869703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980c341-6f40-4911-8078-925f5f0f6962 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:07.869703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980c341-6f40-4911-8078-925f5f0f6962 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:07.871031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 704/1410] 10.222.0.17 () {54 vars in 918 bytes} [Fri Sep 9 22:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:07.876166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2add1-54df-435c-8057-ca2899d03370 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:07.876602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2add1-54df-435c-8057-ca2899d03370 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:07.876693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2add1-54df-435c-8057-ca2899d03370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:07.951896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-35d2add1-54df-435c-8057-ca2899d03370 None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:07.953126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35d2add1-54df-435c-8057-ca2899d03370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:08.056830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 707/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 9 22:56:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:56:08.072243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0b4b5-d2fd-47e8-8f65-ac04da9a34ac None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:08.072565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0b4b5-d2fd-47e8-8f65-ac04da9a34ac None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:08.072842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0b4b5-d2fd-47e8-8f65-ac04da9a34ac None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:08.074157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 705/1412] 10.222.0.17 () {54 vars in 918 bytes} [Fri Sep 9 22:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:08.079182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4634fe5-0e64-41e7-8652-e76d1e402218 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:08.079467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4634fe5-0e64-41e7-8652-e76d1e402218 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:08.079766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4634fe5-0e64-41e7-8652-e76d1e402218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:08.151691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:08.154767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a4634fe5-0e64-41e7-8652-e76d1e402218 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:08.156549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a4634fe5-0e64-41e7-8652-e76d1e402218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:08.269546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 708/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 9 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:10.792925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf954d-9703-4552-85d8-3196cfc48f74 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:10.793457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf954d-9703-4552-85d8-3196cfc48f74 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:10.793457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf954d-9703-4552-85d8-3196cfc48f74 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:10.794784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:10.800246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7a71-305c-44cd-bdc2-12301f75b572 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:10.800670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7a71-305c-44cd-bdc2-12301f75b572 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:10.800786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7a71-305c-44cd-bdc2-12301f75b572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:10.866763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6dbd7a71-305c-44cd-bdc2-12301f75b572 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:10.868001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6dbd7a71-305c-44cd-bdc2-12301f75b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:10.922814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:10.964600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:11.251842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf954d-9703-4552-85d8-3196cfc48f74 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:11.253428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-caaf954d-9703-4552-85d8-3196cfc48f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:11.401753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf954d-9703-4552-85d8-3196cfc48f74 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:11.474659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:11.516131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58baa5c1-00d9-4344-8712-99deb1596e89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:11.516735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baa5c1-00d9-4344-8712-99deb1596e89 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:11.517015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baa5c1-00d9-4344-8712-99deb1596e89 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:11.517284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baa5c1-00d9-4344-8712-99deb1596e89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:11.518897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-58baa5c1-00d9-4344-8712-99deb1596e89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:11.607752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58baa5c1-00d9-4344-8712-99deb1596e89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:11.609356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58baa5c1-00d9-4344-8712-99deb1596e89 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:11.626884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 707/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:12.800873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddff20-08b9-4d53-ab42-b15f32fdf446 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:12.801370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddff20-08b9-4d53-ab42-b15f32fdf446 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:12.801488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddff20-08b9-4d53-ab42-b15f32fdf446 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:12.802767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:12.809003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c90d-e820-470c-bcba-9a767db2cae1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:12.809458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c90d-e820-470c-bcba-9a767db2cae1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:12.809566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6c90d-e820-470c-bcba-9a767db2cae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:12.893885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-87d6c90d-e820-470c-bcba-9a767db2cae1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:12.895134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-87d6c90d-e820-470c-bcba-9a767db2cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:12.924228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:12.993555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:13.142391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddff20-08b9-4d53-ab42-b15f32fdf446 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:13.143973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c5ddff20-08b9-4d53-ab42-b15f32fdf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:13.285764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddff20-08b9-4d53-ab42-b15f32fdf446 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:13.326738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:13.399803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe2d2f-440b-4aac-b33a-ab4e7e828580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:13.401564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe2d2f-440b-4aac-b33a-ab4e7e828580 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:13.401845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe2d2f-440b-4aac-b33a-ab4e7e828580 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:13.402117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe2d2f-440b-4aac-b33a-ab4e7e828580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:13.403752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8bfe2d2f-440b-4aac-b33a-ab4e7e828580 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:13.493414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe2d2f-440b-4aac-b33a-ab4e7e828580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:13.495550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe2d2f-440b-4aac-b33a-ab4e7e828580 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:13.513606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 711/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:14.660005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d32ec0-2679-42b7-9619-265db614a72e None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:14.660592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d32ec0-2679-42b7-9619-265db614a72e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:14.660592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d32ec0-2679-42b7-9619-265db614a72e None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:14.661925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:14.668643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dec5df-15b7-4b48-9cfb-ad7d991714db None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:14.669062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dec5df-15b7-4b48-9cfb-ad7d991714db None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:14.669176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dec5df-15b7-4b48-9cfb-ad7d991714db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:14.724072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:14.747688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e0dec5df-15b7-4b48-9cfb-ad7d991714db None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:14.748909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e0dec5df-15b7-4b48-9cfb-ad7d991714db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:14.844375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:14.997260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d32ec0-2679-42b7-9619-265db614a72e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:14.998794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-29d32ec0-2679-42b7-9619-265db614a72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:15.142726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d32ec0-2679-42b7-9619-265db614a72e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:15.154978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:15.254649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a029a2-138a-4fc7-b1e9-4ccdaaf3fd31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:15.255172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a029a2-138a-4fc7-b1e9-4ccdaaf3fd31 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:15.255449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a029a2-138a-4fc7-b1e9-4ccdaaf3fd31 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:15.255751 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a029a2-138a-4fc7-b1e9-4ccdaaf3fd31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:15.257339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-52a029a2-138a-4fc7-b1e9-4ccdaaf3fd31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:15.345909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a029a2-138a-4fc7-b1e9-4ccdaaf3fd31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:15.347520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a029a2-138a-4fc7-b1e9-4ccdaaf3fd31 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:15.364969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 710/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:16.474512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad67aa-9aed-4f6b-8c84-d8f906a815ae None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:16.475103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad67aa-9aed-4f6b-8c84-d8f906a815ae None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:16.475103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad67aa-9aed-4f6b-8c84-d8f906a815ae None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:16.476409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:16.483014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd0a7e-e8e7-442d-ad79-58234bf7a952 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:16.483436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd0a7e-e8e7-442d-ad79-58234bf7a952 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:16.483674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd0a7e-e8e7-442d-ad79-58234bf7a952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:16.503285 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:16.560234 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cacd0a7e-e8e7-442d-ad79-58234bf7a952 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:16.561443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cacd0a7e-e8e7-442d-ad79-58234bf7a952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:16.656569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:16.805364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad67aa-9aed-4f6b-8c84-d8f906a815ae None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:16.807563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-10ad67aa-9aed-4f6b-8c84-d8f906a815ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:16.915276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:16.951261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad67aa-9aed-4f6b-8c84-d8f906a815ae None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:17.061520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62112e6d-2d6e-4e0d-a93e-f27c5bbb2e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:17.062121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112e6d-2d6e-4e0d-a93e-f27c5bbb2e54 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:17.062401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112e6d-2d6e-4e0d-a93e-f27c5bbb2e54 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:17.062670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112e6d-2d6e-4e0d-a93e-f27c5bbb2e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:17.064299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62112e6d-2d6e-4e0d-a93e-f27c5bbb2e54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:17.139593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:17.154463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62112e6d-2d6e-4e0d-a93e-f27c5bbb2e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:17.156145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62112e6d-2d6e-4e0d-a93e-f27c5bbb2e54 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:17.176838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 714/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:18.268474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f2e5-221d-4297-82bb-f85e5aa96e1c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:18.268960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f2e5-221d-4297-82bb-f85e5aa96e1c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:18.269057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f2e5-221d-4297-82bb-f85e5aa96e1c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:18.270372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:18.277100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a92310-178c-4ef9-a0bf-fb5e36703550 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:18.277528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a92310-178c-4ef9-a0bf-fb5e36703550 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:18.277657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a92310-178c-4ef9-a0bf-fb5e36703550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:18.355309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-07a92310-178c-4ef9-a0bf-fb5e36703550 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:18.356555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-07a92310-178c-4ef9-a0bf-fb5e36703550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:18.450591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:18.606592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72f2e5-221d-4297-82bb-f85e5aa96e1c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:18.608162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7f72f2e5-221d-4297-82bb-f85e5aa96e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:18.692316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:18.754810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72f2e5-221d-4297-82bb-f85e5aa96e1c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:18.870036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f15e3d-c78e-4783-b7ae-77d5df2f2a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:18.870561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f15e3d-c78e-4783-b7ae-77d5df2f2a48 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:18.870841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f15e3d-c78e-4783-b7ae-77d5df2f2a48 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:18.871111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f15e3d-c78e-4783-b7ae-77d5df2f2a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:18.872745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-58f15e3d-c78e-4783-b7ae-77d5df2f2a48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:18.919893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:18.961504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f15e3d-c78e-4783-b7ae-77d5df2f2a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:18.963125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f15e3d-c78e-4783-b7ae-77d5df2f2a48 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:18.980807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 713/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:20.026915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026415a-6b4d-4456-8a3d-490da4ceba71 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:20.027436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026415a-6b4d-4456-8a3d-490da4ceba71 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:20.027563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026415a-6b4d-4456-8a3d-490da4ceba71 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:20.028856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:20.035200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f7d05-dcfd-4eec-b864-4c9edc4763d8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:20.035630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f7d05-dcfd-4eec-b864-4c9edc4763d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:20.035786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f7d05-dcfd-4eec-b864-4c9edc4763d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:20.115263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5c7f7d05-dcfd-4eec-b864-4c9edc4763d8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:20.116516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5c7f7d05-dcfd-4eec-b864-4c9edc4763d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:20.214844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:20.367243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026415a-6b4d-4456-8a3d-490da4ceba71 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:20.368720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5026415a-6b4d-4456-8a3d-490da4ceba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:20.419692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:20.525254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026415a-6b4d-4456-8a3d-490da4ceba71 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:20.635705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fafba-d429-4d70-8be3-d450f0f8a72d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:20.636234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fafba-d429-4d70-8be3-d450f0f8a72d service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:20.636511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fafba-d429-4d70-8be3-d450f0f8a72d service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:20.636782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fafba-d429-4d70-8be3-d450f0f8a72d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:20.638397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6b5fafba-d429-4d70-8be3-d450f0f8a72d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:20.645229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:20.727269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fafba-d429-4d70-8be3-d450f0f8a72d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:20.729440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fafba-d429-4d70-8be3-d450f0f8a72d service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:20.747609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 717/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:21.787769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39169c9b-2bef-4fd8-b078-d36eba574ef8 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:21.788222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39169c9b-2bef-4fd8-b078-d36eba574ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:21.788337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39169c9b-2bef-4fd8-b078-d36eba574ef8 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:21.789630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:21.798702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc6e24-6dba-4ad2-9d51-120df04053ea None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:21.799176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc6e24-6dba-4ad2-9d51-120df04053ea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:21.799273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc6e24-6dba-4ad2-9d51-120df04053ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:21.877003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ddc6e24-6dba-4ad2-9d51-120df04053ea None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:21.878235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ddc6e24-6dba-4ad2-9d51-120df04053ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:21.944946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:21.973898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:22.124071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39169c9b-2bef-4fd8-b078-d36eba574ef8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:22.125646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-39169c9b-2bef-4fd8-b078-d36eba574ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:22.145025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:22.295991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39169c9b-2bef-4fd8-b078-d36eba574ef8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:22.382060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:22.408571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8607450-01e9-48ad-9767-3472d55f425a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:22.409175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8607450-01e9-48ad-9767-3472d55f425a service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:22.409448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8607450-01e9-48ad-9767-3472d55f425a service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:22.409715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8607450-01e9-48ad-9767-3472d55f425a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:22.411302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d8607450-01e9-48ad-9767-3472d55f425a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:22.504361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8607450-01e9-48ad-9767-3472d55f425a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:22.507293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8607450-01e9-48ad-9767-3472d55f425a service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:22.529353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 716/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:23.585466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb303cdd-9790-4914-9d4d-81fbc4d01f32 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:23.585975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb303cdd-9790-4914-9d4d-81fbc4d01f32 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:23.586070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb303cdd-9790-4914-9d4d-81fbc4d01f32 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:23.587365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:23.593450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e585d-175e-4e26-9106-2541d7bc3851 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:23.593909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e585d-175e-4e26-9106-2541d7bc3851 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:23.594017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e585d-175e-4e26-9106-2541d7bc3851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:23.668939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3f7e585d-175e-4e26-9106-2541d7bc3851 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:23.670157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3f7e585d-175e-4e26-9106-2541d7bc3851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:23.715238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:23.766594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:23.917569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb303cdd-9790-4914-9d4d-81fbc4d01f32 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:23.919107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb303cdd-9790-4914-9d4d-81fbc4d01f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:24.062624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb303cdd-9790-4914-9d4d-81fbc4d01f32 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:24.116960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:24.174758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfce414-68f7-4e74-99bb-a5ebe6fe3f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:24.175358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfce414-68f7-4e74-99bb-a5ebe6fe3f97 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:24.175662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfce414-68f7-4e74-99bb-a5ebe6fe3f97 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:24.175933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfce414-68f7-4e74-99bb-a5ebe6fe3f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:24.177508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fcfce414-68f7-4e74-99bb-a5ebe6fe3f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:24.265908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfce414-68f7-4e74-99bb-a5ebe6fe3f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:24.267588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfce414-68f7-4e74-99bb-a5ebe6fe3f97 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:24.286001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 720/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:25.333313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb56ce-4dcf-446a-99ca-e4ac7dc28ded None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:25.333895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb56ce-4dcf-446a-99ca-e4ac7dc28ded None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:25.333895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb56ce-4dcf-446a-99ca-e4ac7dc28ded None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:25.335294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 22:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:25.342050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715582a-6931-49c1-8f3d-aeab2811de66 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:25.342449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715582a-6931-49c1-8f3d-aeab2811de66 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:25.342595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715582a-6931-49c1-8f3d-aeab2811de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:25.417208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e715582a-6931-49c1-8f3d-aeab2811de66 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:25.418403 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e715582a-6931-49c1-8f3d-aeab2811de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:25.420419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:25.513552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 22:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:25.663136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb56ce-4dcf-446a-99ca-e4ac7dc28ded None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:25.664592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-becb56ce-4dcf-446a-99ca-e4ac7dc28ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:25.808545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb56ce-4dcf-446a-99ca-e4ac7dc28ded None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:25.836585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:25.923687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20958480-d1bf-4632-819a-10a0696aad9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:25.924216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958480-d1bf-4632-819a-10a0696aad9f service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:25.924492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958480-d1bf-4632-819a-10a0696aad9f service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:25.924756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958480-d1bf-4632-819a-10a0696aad9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:25.926342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-20958480-d1bf-4632-819a-10a0696aad9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:26.013559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958480-d1bf-4632-819a-10a0696aad9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:26.015132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958480-d1bf-4632-819a-10a0696aad9f service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:26.032444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 719/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:28.089320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd610f7-686f-4e75-be48-08ec8ba496de None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:28.089947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd610f7-686f-4e75-be48-08ec8ba496de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:28.089947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd610f7-686f-4e75-be48-08ec8ba496de None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:28.091187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:28.096557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd20628-50e1-49c5-9829-6cbd522d9f32 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:28.096995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd20628-50e1-49c5-9829-6cbd522d9f32 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:28.097121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd20628-50e1-49c5-9829-6cbd522d9f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:28.137813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:28.163170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0dd20628-50e1-49c5-9829-6cbd522d9f32 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:28.164413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0dd20628-50e1-49c5-9829-6cbd522d9f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:28.259980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:28.546367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd610f7-686f-4e75-be48-08ec8ba496de None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:28.547947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cfd610f7-686f-4e75-be48-08ec8ba496de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:28.649021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd610f7-686f-4e75-be48-08ec8ba496de None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:28.681499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:28.722147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964e1c5-8158-43dd-8811-65ed89123ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:28.722750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964e1c5-8158-43dd-8811-65ed89123ebb service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:28.723026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964e1c5-8158-43dd-8811-65ed89123ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:28.723292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964e1c5-8158-43dd-8811-65ed89123ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:28.724916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9964e1c5-8158-43dd-8811-65ed89123ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:28.833008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964e1c5-8158-43dd-8811-65ed89123ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:28.834383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964e1c5-8158-43dd-8811-65ed89123ebb service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:28.851916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 723/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:31.146901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb11116-3f05-4dd4-acac-c95806f03fd5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:31.147520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb11116-3f05-4dd4-acac-c95806f03fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:31.147520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb11116-3f05-4dd4-acac-c95806f03fd5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:31.148820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:31.154449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820e085-f3f5-46e2-ad66-44b1aa68baf9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:31.154891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820e085-f3f5-46e2-ad66-44b1aa68baf9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:31.155028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820e085-f3f5-46e2-ad66-44b1aa68baf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:31.226795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6820e085-f3f5-46e2-ad66-44b1aa68baf9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:31.228096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6820e085-f3f5-46e2-ad66-44b1aa68baf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:31.240818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:31.324885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:31.615947 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb11116-3f05-4dd4-acac-c95806f03fd5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:31.617484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ecb11116-3f05-4dd4-acac-c95806f03fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:31.721226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb11116-3f05-4dd4-acac-c95806f03fd5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:31.734962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:31.797827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4b84f-7d8e-406c-9b81-bb287c915f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:31.798435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4b84f-7d8e-406c-9b81-bb287c915f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:31.798713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4b84f-7d8e-406c-9b81-bb287c915f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:31.798980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4b84f-7d8e-406c-9b81-bb287c915f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:31.800896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f7f4b84f-7d8e-406c-9b81-bb287c915f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:31.910712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4b84f-7d8e-406c-9b81-bb287c915f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:31.912072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4b84f-7d8e-406c-9b81-bb287c915f1c service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:31.930267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 722/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:34.387252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3652af-0deb-4f8b-ba39-a58d0e035b55 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:34.387858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3652af-0deb-4f8b-ba39-a58d0e035b55 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:34.387858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3652af-0deb-4f8b-ba39-a58d0e035b55 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:34.389186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:34.394561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a612e-90b7-48b7-b7d0-c0b14514b9b5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:34.394976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a612e-90b7-48b7-b7d0-c0b14514b9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:34.395113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a612e-90b7-48b7-b7d0-c0b14514b9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:34.443144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:34.460722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-275a612e-90b7-48b7-b7d0-c0b14514b9b5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:34.461958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-275a612e-90b7-48b7-b7d0-c0b14514b9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:34.559311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:34.846540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3652af-0deb-4f8b-ba39-a58d0e035b55 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:34.848201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3d3652af-0deb-4f8b-ba39-a58d0e035b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:34.952032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3652af-0deb-4f8b-ba39-a58d0e035b55 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:34.991931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:35.026943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d85cc-d40d-4fe0-962c-d5d88b38b8cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:35.027579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d85cc-d40d-4fe0-962c-d5d88b38b8cf service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:35.027861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d85cc-d40d-4fe0-962c-d5d88b38b8cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:35.028130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d85cc-d40d-4fe0-962c-d5d88b38b8cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:35.029733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a28d85cc-d40d-4fe0-962c-d5d88b38b8cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:35.138295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d85cc-d40d-4fe0-962c-d5d88b38b8cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:35.139782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d85cc-d40d-4fe0-962c-d5d88b38b8cf service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:35.164249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 726/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:37.474189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd7801-102b-4028-bd9c-57516c9ac38d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:37.474768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd7801-102b-4028-bd9c-57516c9ac38d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:37.474768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd7801-102b-4028-bd9c-57516c9ac38d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:37.476134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:37.482130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d0d3a-8e86-46e8-8253-118134a4fabb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:37.482547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d0d3a-8e86-46e8-8253-118134a4fabb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:37.482703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d0d3a-8e86-46e8-8253-118134a4fabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:37.553003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c83d0d3a-8e86-46e8-8253-118134a4fabb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:37.554254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c83d0d3a-8e86-46e8-8253-118134a4fabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:37.578908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:37.660093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:56:37.971903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd7801-102b-4028-bd9c-57516c9ac38d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:37.973448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-09bd7801-102b-4028-bd9c-57516c9ac38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:38.080085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd7801-102b-4028-bd9c-57516c9ac38d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:38.100815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:38.153595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edfb1a-7790-4d70-922c-0c730d3cc32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:38.154194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edfb1a-7790-4d70-922c-0c730d3cc32d service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:38.154471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edfb1a-7790-4d70-922c-0c730d3cc32d service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:38.154738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edfb1a-7790-4d70-922c-0c730d3cc32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:38.156347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f0edfb1a-7790-4d70-922c-0c730d3cc32d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:38.269888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edfb1a-7790-4d70-922c-0c730d3cc32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:38.271314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edfb1a-7790-4d70-922c-0c730d3cc32d service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:38.289176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 725/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:40.464376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf408-743c-4519-93f0-688b64b61db1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:40.464952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf408-743c-4519-93f0-688b64b61db1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:40.464952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf408-743c-4519-93f0-688b64b61db1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:40.466256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:40.471697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0062030-9f65-4693-a987-a0a6f07431dc None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:40.472136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0062030-9f65-4693-a987-a0a6f07431dc None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:40.472278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0062030-9f65-4693-a987-a0a6f07431dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:40.535681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:40.538267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a0062030-9f65-4693-a987-a0a6f07431dc None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:40.540779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a0062030-9f65-4693-a987-a0a6f07431dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:40.634925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:40.917586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bf408-743c-4519-93f0-688b64b61db1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:40.919131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3d8bf408-743c-4519-93f0-688b64b61db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:41.021356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bf408-743c-4519-93f0-688b64b61db1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:41.069551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:41.093916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab679460-31cf-4137-989e-e53901724b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:41.094512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab679460-31cf-4137-989e-e53901724b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:41.094789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab679460-31cf-4137-989e-e53901724b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:41.095055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab679460-31cf-4137-989e-e53901724b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:41.096649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab679460-31cf-4137-989e-e53901724b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:41.209066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab679460-31cf-4137-989e-e53901724b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:41.210503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab679460-31cf-4137-989e-e53901724b7e service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:41.228279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 729/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:43.419458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f430a6-1dab-4823-ada8-792381887f73 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:43.420066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f430a6-1dab-4823-ada8-792381887f73 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:43.420066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f430a6-1dab-4823-ada8-792381887f73 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:43.421602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:43.426809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10551e-ee86-4b46-8222-2fa4984ff85d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:43.427209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10551e-ee86-4b46-8222-2fa4984ff85d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:43.427356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10551e-ee86-4b46-8222-2fa4984ff85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:43.491710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e10551e-ee86-4b46-8222-2fa4984ff85d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:43.492919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e10551e-ee86-4b46-8222-2fa4984ff85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:43.521303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:43.589423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:43.887185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f430a6-1dab-4823-ada8-792381887f73 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:43.888779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f5f430a6-1dab-4823-ada8-792381887f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:43.989640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f430a6-1dab-4823-ada8-792381887f73 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:44.017443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:44.062190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3004174-cd07-4d44-b6bf-93ab190f0334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:44.062791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3004174-cd07-4d44-b6bf-93ab190f0334 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:44.063067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3004174-cd07-4d44-b6bf-93ab190f0334 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:44.063326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3004174-cd07-4d44-b6bf-93ab190f0334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:44.065896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3004174-cd07-4d44-b6bf-93ab190f0334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:44.173187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3004174-cd07-4d44-b6bf-93ab190f0334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:44.174670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3004174-cd07-4d44-b6bf-93ab190f0334 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:44.193268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 728/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:46.410613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317d7e7-5417-4a4c-b711-6f32f750d846 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:46.411219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317d7e7-5417-4a4c-b711-6f32f750d846 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:46.411219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317d7e7-5417-4a4c-b711-6f32f750d846 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:46.412543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:46.417808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c16a4-9732-496d-81d2-07d6e5ade2c7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:46.418219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c16a4-9732-496d-81d2-07d6e5ade2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:46.418381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c16a4-9732-496d-81d2-07d6e5ade2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:46.483769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-095c16a4-9732-496d-81d2-07d6e5ade2c7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:46.484998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-095c16a4-9732-496d-81d2-07d6e5ade2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:46.493649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:46.580502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:46.862334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317d7e7-5417-4a4c-b711-6f32f750d846 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:46.863904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b317d7e7-5417-4a4c-b711-6f32f750d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:46.968095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317d7e7-5417-4a4c-b711-6f32f750d846 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:47.024038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:47.043622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf174-5dd6-4773-a05e-af561e2b5839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:47.044231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf174-5dd6-4773-a05e-af561e2b5839 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:47.044508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf174-5dd6-4773-a05e-af561e2b5839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:47.044777 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf174-5dd6-4773-a05e-af561e2b5839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:47.046658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e3daf174-5dd6-4773-a05e-af561e2b5839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:47.156381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3daf174-5dd6-4773-a05e-af561e2b5839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:47.158119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3daf174-5dd6-4773-a05e-af561e2b5839 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:47.178679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 732/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:49.357835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbbe8a-b2be-44fc-8216-f7b9ac77af23 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:49.358409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbbe8a-b2be-44fc-8216-f7b9ac77af23 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:49.358409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbbe8a-b2be-44fc-8216-f7b9ac77af23 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:49.359774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:49.364984 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06e4b8-b532-4c95-8f43-64f3b6838977 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:49.365420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06e4b8-b532-4c95-8f43-64f3b6838977 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:49.365532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06e4b8-b532-4c95-8f43-64f3b6838977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:49.433288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fa06e4b8-b532-4c95-8f43-64f3b6838977 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:49.434503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fa06e4b8-b532-4c95-8f43-64f3b6838977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:49.473076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:49.530734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:49.812528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbbe8a-b2be-44fc-8216-f7b9ac77af23 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:49.814067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-08cbbe8a-b2be-44fc-8216-f7b9ac77af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:49.916492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbbe8a-b2be-44fc-8216-f7b9ac77af23 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:49.956210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:49.994568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06764cb5-3cd2-4ea8-9fbe-03c700434d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:49.995120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764cb5-3cd2-4ea8-9fbe-03c700434d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:49.995403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764cb5-3cd2-4ea8-9fbe-03c700434d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:49.995714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06764cb5-3cd2-4ea8-9fbe-03c700434d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:49.997319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-06764cb5-3cd2-4ea8-9fbe-03c700434d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:50.110055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06764cb5-3cd2-4ea8-9fbe-03c700434d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:50.111556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06764cb5-3cd2-4ea8-9fbe-03c700434d1c service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:50.131883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 731/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:52.369661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46286bd1-7931-4e65-9d78-b6effcc5e0d5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:52.370213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46286bd1-7931-4e65-9d78-b6effcc5e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:52.370213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46286bd1-7931-4e65-9d78-b6effcc5e0d5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:52.371584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:52.376723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae418d-ff86-454c-ab5b-ab1f9453810b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:52.377133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae418d-ff86-454c-ab5b-ab1f9453810b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:52.377268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae418d-ff86-454c-ab5b-ab1f9453810b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:52.441888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0cae418d-ff86-454c-ab5b-ab1f9453810b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:52.443076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0cae418d-ff86-454c-ab5b-ab1f9453810b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:52.461165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:52.538288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:52.849569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46286bd1-7931-4e65-9d78-b6effcc5e0d5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:52.851090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46286bd1-7931-4e65-9d78-b6effcc5e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:52.951988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46286bd1-7931-4e65-9d78-b6effcc5e0d5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:53.020256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:53.028072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71979cb-f91f-486b-9e0b-7d5c4dc7cae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:53.028688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71979cb-f91f-486b-9e0b-7d5c4dc7cae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:53.028965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71979cb-f91f-486b-9e0b-7d5c4dc7cae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:53.029235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71979cb-f91f-486b-9e0b-7d5c4dc7cae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:53.030817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b71979cb-f91f-486b-9e0b-7d5c4dc7cae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:53.142975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71979cb-f91f-486b-9e0b-7d5c4dc7cae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:53.144455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71979cb-f91f-486b-9e0b-7d5c4dc7cae1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:53.277431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 735/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:56:55.421731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024992e-9581-4d1c-bfd1-e4066b2f0abb None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:55.422284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024992e-9581-4d1c-bfd1-e4066b2f0abb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:55.422284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024992e-9581-4d1c-bfd1-e4066b2f0abb None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:55.423671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 22:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:56:55.429112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db38812-cd7a-4c8c-8eb8-7d214d221187 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:55.429530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db38812-cd7a-4c8c-8eb8-7d214d221187 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:55.429649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db38812-cd7a-4c8c-8eb8-7d214d221187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:55.493985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9db38812-cd7a-4c8c-8eb8-7d214d221187 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:56:55.495170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9db38812-cd7a-4c8c-8eb8-7d214d221187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:55.542163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:55.589828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 22:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:56:55.869077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024992e-9581-4d1c-bfd1-e4066b2f0abb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:56:55.870603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c024992e-9581-4d1c-bfd1-e4066b2f0abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:55.971780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024992e-9581-4d1c-bfd1-e4066b2f0abb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:56:56.020577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:56:56.046746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f4be1-b73e-447e-a24b-ebcc64aa70cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:56:56.047343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f4be1-b73e-447e-a24b-ebcc64aa70cb service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:56:56.047653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f4be1-b73e-447e-a24b-ebcc64aa70cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:56:56.047924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f4be1-b73e-447e-a24b-ebcc64aa70cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:56:56.049500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9c2f4be1-b73e-447e-a24b-ebcc64aa70cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:56:56.159170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f4be1-b73e-447e-a24b-ebcc64aa70cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:56:56.160638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f4be1-b73e-447e-a24b-ebcc64aa70cb service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:56:56.294766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 734/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:57:05.588342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=22 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:06.292166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=66 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:16.200232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ba72-1f6c-4984-a871-7eaf5c7590e8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:16.200808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ba72-1f6c-4984-a871-7eaf5c7590e8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:16.200808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ba72-1f6c-4984-a871-7eaf5c7590e8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:16.202102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 737/1471] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:16.207236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ac280-0bb6-41d0-bf82-4ea7f854e145 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:16.207712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ac280-0bb6-41d0-bf82-4ea7f854e145 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:16.207816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ac280-0bb6-41d0-bf82-4ea7f854e145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:16.275672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c32ac280-0bb6-41d0-bf82-4ea7f854e145 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:16.276946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c32ac280-0bb6-41d0-bf82-4ea7f854e145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:16.375099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 735/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:57:16.674661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bb5ac-cd3c-4a1d-89fa-64f9a2ef790d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:16.674941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bb5ac-cd3c-4a1d-89fa-64f9a2ef790d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:16.675208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bb5ac-cd3c-4a1d-89fa-64f9a2ef790d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:16.676580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 738/1473] 10.222.0.17 () {54 vars in 966 bytes} [Fri Sep 9 22:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:16.681511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a1eb-1140-4ec3-a144-0662f667164b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:16.681794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a1eb-1140-4ec3-a144-0662f667164b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:16.682062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a1eb-1140-4ec3-a144-0662f667164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:16.756652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b553a1eb-1140-4ec3-a144-0662f667164b None None] MFA Rules not processed for user `6cf808a1f5dd45bea0a3a81131532b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:16.758383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b553a1eb-1140-4ec3-a144-0662f667164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:16.779169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:16.835446 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 736/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Fri Sep 9 22:57:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 22:57:16.843121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bb5ac-cd3c-4a1d-89fa-64f9a2ef790d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:57:16.844668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a96bb5ac-cd3c-4a1d-89fa-64f9a2ef790d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:16.950026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bb5ac-cd3c-4a1d-89fa-64f9a2ef790d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:57:17.022685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78889ebb-8929-4878-8f36-73e8be0b63cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:57:17.023414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78889ebb-8929-4878-8f36-73e8be0b63cd service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:17.023727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78889ebb-8929-4878-8f36-73e8be0b63cd service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:17.024000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78889ebb-8929-4878-8f36-73e8be0b63cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:17.025612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78889ebb-8929-4878-8f36-73e8be0b63cd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:17.080300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:17.134869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78889ebb-8929-4878-8f36-73e8be0b63cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:57:17.136290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78889ebb-8929-4878-8f36-73e8be0b63cd service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:57:17.154824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 739/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:57:19.386349 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93e322-c806-48bf-acda-cdf844cb3381 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:19.386923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93e322-c806-48bf-acda-cdf844cb3381 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:19.386923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93e322-c806-48bf-acda-cdf844cb3381 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:19.388256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 737/1476] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:19.393486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079f70e-387f-41b8-b4a2-1c762985eddf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:19.393960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079f70e-387f-41b8-b4a2-1c762985eddf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:19.394065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079f70e-387f-41b8-b4a2-1c762985eddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:19.459115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e079f70e-387f-41b8-b4a2-1c762985eddf None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:19.460331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e079f70e-387f-41b8-b4a2-1c762985eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:19.555036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 740/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:57:19.838401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93e322-c806-48bf-acda-cdf844cb3381 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:57:19.839979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ac93e322-c806-48bf-acda-cdf844cb3381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:19.963250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93e322-c806-48bf-acda-cdf844cb3381 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:57:20.013204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:20.057453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5687dab6-734c-4206-86df-0c9a4c7b9aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:57:20.058150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687dab6-734c-4206-86df-0c9a4c7b9aad admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:20.058433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687dab6-734c-4206-86df-0c9a4c7b9aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:20.058700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687dab6-734c-4206-86df-0c9a4c7b9aad admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:20.059328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5687dab6-734c-4206-86df-0c9a4c7b9aad admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:57:20.061126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-5687dab6-734c-4206-86df-0c9a4c7b9aad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:57:20.061126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 22:57:20.061126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 22:57:20.062268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5687dab6-734c-4206-86df-0c9a4c7b9aad admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:57:20.083054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 738/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 22:57:19 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 22:57:22.260273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782bb73-c07f-43c2-a259-dd1aec7d409a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:22.260869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782bb73-c07f-43c2-a259-dd1aec7d409a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:22.260869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782bb73-c07f-43c2-a259-dd1aec7d409a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:22.262231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 741/1479] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:22.267381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c64d7a-3930-47e9-bde4-1c5c1a30f85f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:22.267829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c64d7a-3930-47e9-bde4-1c5c1a30f85f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:22.267952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c64d7a-3930-47e9-bde4-1c5c1a30f85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:22.341489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c4c64d7a-3930-47e9-bde4-1c5c1a30f85f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:22.342709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c4c64d7a-3930-47e9-bde4-1c5c1a30f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:22.438123 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 739/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:57:22.735258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782bb73-c07f-43c2-a259-dd1aec7d409a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:57:22.737207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f782bb73-c07f-43c2-a259-dd1aec7d409a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:22.747022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:22.879864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782bb73-c07f-43c2-a259-dd1aec7d409a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:57:22.958208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:22.990878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105697ef-ccc5-41a5-818a-6fc2f85d2f26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:57:22.991420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105697ef-ccc5-41a5-818a-6fc2f85d2f26 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:22.991722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105697ef-ccc5-41a5-818a-6fc2f85d2f26 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:22.991991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105697ef-ccc5-41a5-818a-6fc2f85d2f26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:22.993575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-105697ef-ccc5-41a5-818a-6fc2f85d2f26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:23.082065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105697ef-ccc5-41a5-818a-6fc2f85d2f26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:57:23.083693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105697ef-ccc5-41a5-818a-6fc2f85d2f26 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:57:23.101575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 742/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:57:25.427980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ccf5-a845-484a-81ad-3eafc6233095 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:25.428557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ccf5-a845-484a-81ad-3eafc6233095 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:25.428557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ccf5-a845-484a-81ad-3eafc6233095 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:25.429843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 740/1482] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:25.435164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20a91-e70a-4c47-84e5-f79f858c124e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:25.435624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20a91-e70a-4c47-84e5-f79f858c124e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:25.435730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20a91-e70a-4c47-84e5-f79f858c124e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:25.500247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-28f20a91-e70a-4c47-84e5-f79f858c124e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:25.501633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-28f20a91-e70a-4c47-84e5-f79f858c124e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:25.547162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:25.596852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 743/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:57:25.885458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4ccf5-a845-484a-81ad-3eafc6233095 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:57:25.886977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b7e4ccf5-a845-484a-81ad-3eafc6233095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:25.910756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:25.989989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4ccf5-a845-484a-81ad-3eafc6233095 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:57:26.062719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd8ef7-e7b3-408d-a22e-f97414baea44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:57:26.063315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd8ef7-e7b3-408d-a22e-f97414baea44 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:26.063619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd8ef7-e7b3-408d-a22e-f97414baea44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:26.063886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd8ef7-e7b3-408d-a22e-f97414baea44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:26.065488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-14dd8ef7-e7b3-408d-a22e-f97414baea44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:26.137895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:26.172923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd8ef7-e7b3-408d-a22e-f97414baea44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:57:26.174332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd8ef7-e7b3-408d-a22e-f97414baea44 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:57:26.192557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 741/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:57:28.371121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bb39f-d031-484c-afc5-60c11e562535 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:28.371732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bb39f-d031-484c-afc5-60c11e562535 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:28.371732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bb39f-d031-484c-afc5-60c11e562535 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:28.373067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 744/1485] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:28.378336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8a70-169a-42a6-a4a4-7f3dc554c4f2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:28.378742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8a70-169a-42a6-a4a4-7f3dc554c4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:28.378872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8a70-169a-42a6-a4a4-7f3dc554c4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:28.444138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a64a8a70-169a-42a6-a4a4-7f3dc554c4f2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:28.445356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a64a8a70-169a-42a6-a4a4-7f3dc554c4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:28.539166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 742/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:57:28.828153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bb39f-d031-484c-afc5-60c11e562535 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:57:28.829695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8a8bb39f-d031-484c-afc5-60c11e562535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:28.931147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bb39f-d031-484c-afc5-60c11e562535 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:57:29.001439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:29.002930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f440c7-9f75-4f6d-832a-5b062c4d3999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:57:29.003546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f440c7-9f75-4f6d-832a-5b062c4d3999 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:29.004026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f440c7-9f75-4f6d-832a-5b062c4d3999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:29.004517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f440c7-9f75-4f6d-832a-5b062c4d3999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:29.006345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-31f440c7-9f75-4f6d-832a-5b062c4d3999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:29.113158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f440c7-9f75-4f6d-832a-5b062c4d3999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:57:29.114581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f440c7-9f75-4f6d-832a-5b062c4d3999 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:57:29.132699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 745/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:57:31.480909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6ad3-0995-4ba1-89ba-eb1fb036348b None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:31.481512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6ad3-0995-4ba1-89ba-eb1fb036348b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:31.481512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6ad3-0995-4ba1-89ba-eb1fb036348b None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:31.482766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 743/1488] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:31.488228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c4fd-8736-4850-8632-868be4cd9f5e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:31.488727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c4fd-8736-4850-8632-868be4cd9f5e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:31.488830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c4fd-8736-4850-8632-868be4cd9f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:31.555829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-68a4c4fd-8736-4850-8632-868be4cd9f5e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:31.557061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68a4c4fd-8736-4850-8632-868be4cd9f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:31.609298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:31.652395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 746/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:57:31.950161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be6ad3-0995-4ba1-89ba-eb1fb036348b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:57:31.951711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e3be6ad3-0995-4ba1-89ba-eb1fb036348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:31.987119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:32.054636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be6ad3-0995-4ba1-89ba-eb1fb036348b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:57:32.136042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4b78-695c-4b11-b2c6-5b3fc940f68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:57:32.136649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4b78-695c-4b11-b2c6-5b3fc940f68c service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:32.136928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4b78-695c-4b11-b2c6-5b3fc940f68c service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:32.137200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4b78-695c-4b11-b2c6-5b3fc940f68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:32.138813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d35e4b78-695c-4b11-b2c6-5b3fc940f68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:32.219545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:32.248640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4b78-695c-4b11-b2c6-5b3fc940f68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:57:32.249991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4b78-695c-4b11-b2c6-5b3fc940f68c service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:57:32.267927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 744/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:57:34.639636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b909bd-2b25-4982-947a-6449a3cc28b3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:34.640255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b909bd-2b25-4982-947a-6449a3cc28b3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:34.640255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b909bd-2b25-4982-947a-6449a3cc28b3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:34.641530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 747/1491] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 22:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:57:34.646688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05aded-9912-4c90-b3cf-63be67554740 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:34.647121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05aded-9912-4c90-b3cf-63be67554740 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:34.647216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05aded-9912-4c90-b3cf-63be67554740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:34.714189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7e05aded-9912-4c90-b3cf-63be67554740 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:57:34.715390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7e05aded-9912-4c90-b3cf-63be67554740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:34.810929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 745/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 22:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:57:35.107022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b909bd-2b25-4982-947a-6449a3cc28b3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:57:35.108592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-79b909bd-2b25-4982-947a-6449a3cc28b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:35.212370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b909bd-2b25-4982-947a-6449a3cc28b3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:57:35.283963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2fac9-4761-4054-b488-576c607a4d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:57:35.284568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2fac9-4761-4054-b488-576c607a4d88 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:57:35.284844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2fac9-4761-4054-b488-576c607a4d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:57:35.285106 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2fac9-4761-4054-b488-576c607a4d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:57:35.286677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-edc2fac9-4761-4054-b488-576c607a4d88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:57:35.296347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:35.401723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2fac9-4761-4054-b488-576c607a4d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:57:35.403072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2fac9-4761-4054-b488-576c607a4d88 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:57:35.422111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 748/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:57:44.809437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=82 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:57:45.419566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=50 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:54.043795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:54.044337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:54.044337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:54.119302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:58:54.120548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:54.216034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 746/1494] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 22:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:58:54.224198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae9a3-b5f5-4bb6-a4ec-1a94ddbc77ef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:54.224683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae9a3-b5f5-4bb6-a4ec-1a94ddbc77ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:54.224793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae9a3-b5f5-4bb6-a4ec-1a94ddbc77ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:54.298433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dc1ae9a3-b5f5-4bb6-a4ec-1a94ddbc77ef None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:58:54.299586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc1ae9a3-b5f5-4bb6-a4ec-1a94ddbc77ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:54.394442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 749/1495] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 22:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:58:54.398997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:54.400228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:54.412060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a07bf3-d6e6-4adc-928e-1fdfca2bef42 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:54.473229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:54.506785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:54.507386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:54.507693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:54.507963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:54.508595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 22:58:54.508890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:54.509887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:54.509887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 22:58:54.509887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 22:58:54.510762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:54.521790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 747/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 22:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 22:58:54.530037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ae9a3-b5f5-4bb6-a4ec-1a94ddbc77ef None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:54.539669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ae9a3-b5f5-4bb6-a4ec-1a94ddbc77ef None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:54.600116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:54.634417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:54.635032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:54.635310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:54.635610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:54.636253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:54.637990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:54.637990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 09 22:58:54.637990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: Sep 09 22:58:54.639179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:54.696579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 750/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 22:58:54 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:58:54.700808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:54.702463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:54.737423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24536-7be4-4b1f-b526-0a47e3b2047f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:54.830746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:54.831347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:54.831656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:54.831924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:54.834498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:54.835524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:54.835524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 22:58:54.835524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 22:58:54.836724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:54.878407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 748/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 22:58:54 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 22:58:54.883023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:54.884567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:54.945187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c5d4c-646c-496f-81f3-02084fa0982c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:55.041077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:55.053608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:55.054216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:55.054492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:55.054784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:55.055457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:55.056494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:55.056494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 09 22:58:55.056494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: Sep 09 22:58:55.057373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:55.069605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 751/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 22:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 22:58:55.073591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:55.075091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:55.086215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d3c1-83c7-4f9c-b7b4-7d627afc8e38 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:55.111781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:55.181433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd93f9-d152-4cda-bdf7-01eb72958858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:55.181964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd93f9-d152-4cda-bdf7-01eb72958858 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:55.182243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd93f9-d152-4cda-bdf7-01eb72958858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:55.182515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd93f9-d152-4cda-bdf7-01eb72958858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:55.183157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd93f9-d152-4cda-bdf7-01eb72958858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:55.184193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-10fd93f9-d152-4cda-bdf7-01eb72958858 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:55.184193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 22:58:55.184193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 22:58:55.185074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd93f9-d152-4cda-bdf7-01eb72958858 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:55.197587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 749/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 22:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 22:58:55.201792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:55.203322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:55.236400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010ae9e-38cc-4d17-809b-351c9f64fee0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:55.331333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3e1ce-4b15-42fb-b6cb-12235ad0a45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:55.332018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3e1ce-4b15-42fb-b6cb-12235ad0a45e admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:55.332893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3e1ce-4b15-42fb-b6cb-12235ad0a45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:55.333165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3e1ce-4b15-42fb-b6cb-12235ad0a45e admin admin] PATH_INFO: `/v3/projects/e18d5e85798343ae8c0cd47296c6212c/users/1c3387c7eb2549fdb5bbff4e3df5e2f6/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:55.355252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3e1ce-4b15-42fb-b6cb-12235ad0a45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18d5e85798343ae8c0cd47296c6212c, user_id=1c3387c7eb2549fdb5bbff4e3df5e2f6, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:55.356321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-18a3e1ce-4b15-42fb-b6cb-12235ad0a45e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:55.356321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 09 22:58:55.356321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: Sep 09 22:58:55.358251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3e1ce-4b15-42fb-b6cb-12235ad0a45e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:55.382483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 752/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 22:58:55 2022] PUT /identity/v3/projects/e18d5e85798343ae8c0cd47296c6212c/users/1c3387c7eb2549fdb5bbff4e3df5e2f6/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:58:55.394680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a3b7e-8b4e-4c21-91bd-7ab30c67596b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:55.394992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a3b7e-8b4e-4c21-91bd-7ab30c67596b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:55.395867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a3b7e-8b4e-4c21-91bd-7ab30c67596b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:55.452544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-173a3b7e-8b4e-4c21-91bd-7ab30c67596b None None] MFA Rules not processed for user `1c3387c7eb2549fdb5bbff4e3df5e2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:58:55.453692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-173a3b7e-8b4e-4c21-91bd-7ab30c67596b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:55.529705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 750/1502] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 22:58:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:58:55.542845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5da3d-e4e8-4586-a8e6-3074eefbc685 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:55.543134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5da3d-e4e8-4586-a8e6-3074eefbc685 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:55.543406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5da3d-e4e8-4586-a8e6-3074eefbc685 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:55.544795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 753/1503] 10.222.0.17 () {54 vars in 964 bytes} [Fri Sep 9 22:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:58:55.549755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98373375-7f53-4558-b75d-45c624f1912e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:55.550048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98373375-7f53-4558-b75d-45c624f1912e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:55.550325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98373375-7f53-4558-b75d-45c624f1912e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:55.583609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:55.627755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-98373375-7f53-4558-b75d-45c624f1912e None None] MFA Rules not processed for user `56228e09c8b742b29e49c40d1a39870f`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:58:55.628981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-98373375-7f53-4558-b75d-45c624f1912e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:55.705487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 751/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Sep 9 22:58:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:58:55.713535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5da3d-e4e8-4586-a8e6-3074eefbc685 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:55.715041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-12c5da3d-e4e8-4586-a8e6-3074eefbc685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:55.765005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:55.819380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5da3d-e4e8-4586-a8e6-3074eefbc685 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:55.893117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:55.893732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:55.894012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:55.894281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:55.896558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:55.992490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:55.993844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:55.997271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:56.012438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 754/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:58:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:58:56.057070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f91058-3598-4693-ae9f-aa3f538d2999 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:56.057383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f91058-3598-4693-ae9f-aa3f538d2999 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:56.057656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f91058-3598-4693-ae9f-aa3f538d2999 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:56.058895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 752/1506] 10.222.0.17 () {52 vars in 830 bytes} [Fri Sep 9 22:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 22:58:56.062399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:56.063629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:56.159728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2523288-daec-4b71-bdfc-eae96a23a6bf service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:56.249179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:56.249815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:56.249994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:56.250264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] PATH_INFO: `/v3/users/1c3387c7eb2549fdb5bbff4e3df5e2f6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:56.263143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3387c7eb2549fdb5bbff4e3df5e2f6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:56.264194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:56.264194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 09 22:58:56.264194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: Sep 09 22:58:56.265389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:56.274675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:56.330390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3387c7eb2549fdb5bbff4e3df5e2f6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:56.344827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3387c7eb2549fdb5bbff4e3df5e2f6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:56.368358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3387c7eb2549fdb5bbff4e3df5e2f6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:56.368924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3387c7eb2549fdb5bbff4e3df5e2f6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:56.369348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3387c7eb2549fdb5bbff4e3df5e2f6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:56.383936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 755/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 22:58:56 2022] DELETE /identity/v3/users/1c3387c7eb2549fdb5bbff4e3df5e2f6 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:58:56.388244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:56.388700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:56.388817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:56.445366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 22:58:56.446569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:56.521539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:56.534093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 753/1508] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 22:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 22:58:56.543769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:56.545022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:56.644565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8628c6-c4cb-483e-b38e-44ca9d04b80c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:56.701363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:58:56.723096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae21d8-1402-45c3-bd7c-a325fef8e46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:56.723771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae21d8-1402-45c3-bd7c-a325fef8e46e service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:56.724052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae21d8-1402-45c3-bd7c-a325fef8e46e service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:56.724322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae21d8-1402-45c3-bd7c-a325fef8e46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:56.725922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7eae21d8-1402-45c3-bd7c-a325fef8e46e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:56.833848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae21d8-1402-45c3-bd7c-a325fef8e46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:56.835204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae21d8-1402-45c3-bd7c-a325fef8e46e service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:56.852545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 756/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 22:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 22:58:57.196055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 22:58:57.197274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 22:58:57.208430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6816e7a-1066-47bb-ab18-a378fb9522eb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 22:58:57.305972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 22:58:57.306599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 22:58:57.306936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 22:58:57.307211 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] PATH_INFO: `/v3/projects/e18d5e85798343ae8c0cd47296c6212c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 22:58:57.316997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18d5e85798343ae8c0cd47296c6212c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 22:58:57.318022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 22:58:57.318022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 22:58:57.318022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 22:58:57.319208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 22:58:57.380476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18d5e85798343ae8c0cd47296c6212c', 'request_id': 'req-67a5ac37-546b-442f-b1aa-597873c23a65'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:57.393370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18d5e85798343ae8c0cd47296c6212c', 'request_id': 'req-67a5ac37-546b-442f-b1aa-597873c23a65'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:57.401763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18d5e85798343ae8c0cd47296c6212c', 'request_id': 'req-67a5ac37-546b-442f-b1aa-597873c23a65'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:57.424856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] The token cache is being invalidate because project e18d5e85798343ae8c0cd47296c6212c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 22:58:57.425151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:57.425709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67a5ac37-546b-442f-b1aa-597873c23a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 22:58:57.427690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 754/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 22:58:57 2022] DELETE /identity/v3/projects/e18d5e85798343ae8c0cd47296c6212c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 22:59:06.850294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=62 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 22:59:07.422801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:00:25.048771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d29a4-cc15-468b-abea-1379a976b9f6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:25.049361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d29a4-cc15-468b-abea-1379a976b9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:25.049361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d29a4-cc15-468b-abea-1379a976b9f6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:25.050676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:25.055456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aff3c-d01d-4007-8498-853f725fc28d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:25.055921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aff3c-d01d-4007-8498-853f725fc28d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:25.056019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aff3c-d01d-4007-8498-853f725fc28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:25.136092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d51aff3c-d01d-4007-8498-853f725fc28d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:25.137296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d51aff3c-d01d-4007-8498-853f725fc28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:25.239365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:25.394711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d29a4-cc15-468b-abea-1379a976b9f6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:25.396915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d07d29a4-cc15-468b-abea-1379a976b9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:25.514148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d29a4-cc15-468b-abea-1379a976b9f6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:25.520274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce04714-fbeb-4a84-af11-70893088ee66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:25.520882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce04714-fbeb-4a84-af11-70893088ee66 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:25.521196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce04714-fbeb-4a84-af11-70893088ee66 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:25.521493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce04714-fbeb-4a84-af11-70893088ee66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:25.529451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce04714-fbeb-4a84-af11-70893088ee66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:25.531037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce04714-fbeb-4a84-af11-70893088ee66 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:25.540267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 758/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:26.445053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a618f-f93d-431c-9bc1-1f256ff34c65 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:26.445652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a618f-f93d-431c-9bc1-1f256ff34c65 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:26.445652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a618f-f93d-431c-9bc1-1f256ff34c65 None None] PATH_INFO: `/v3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:26.446875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 756/1514] 10.222.0.15 () {54 vars in 925 bytes} [Fri Sep 9 23:00:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:00:26.453052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bec1a-064d-4eec-adc4-d3a0589b2670 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:26.453536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bec1a-064d-4eec-adc4-d3a0589b2670 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:26.453643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bec1a-064d-4eec-adc4-d3a0589b2670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:26.513053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-338bec1a-064d-4eec-adc4-d3a0589b2670 None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:26.514260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-338bec1a-064d-4eec-adc4-d3a0589b2670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:26.583517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:00:26.589357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 759/1515] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 23:00:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:00:26.625031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea166a-5ae3-41b0-b845-9b25aaf2354c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:26.625319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea166a-5ae3-41b0-b845-9b25aaf2354c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:26.625595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea166a-5ae3-41b0-b845-9b25aaf2354c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:26.626909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 757/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:26.632911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcccffd-a514-4e25-9a22-f8b9d8b1327d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:26.633198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcccffd-a514-4e25-9a22-f8b9d8b1327d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:26.633473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcccffd-a514-4e25-9a22-f8b9d8b1327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:26.652374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7fcccffd-a514-4e25-9a22-f8b9d8b1327d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:26.653562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7fcccffd-a514-4e25-9a22-f8b9d8b1327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:26.661788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 760/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:26.812270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea166a-5ae3-41b0-b845-9b25aaf2354c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:26.826806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea166a-5ae3-41b0-b845-9b25aaf2354c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:26.832275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a979d44-00b2-46a7-a570-edf5faf77784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:26.833361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a979d44-00b2-46a7-a570-edf5faf77784 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:26.833642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a979d44-00b2-46a7-a570-edf5faf77784 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:26.833915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a979d44-00b2-46a7-a570-edf5faf77784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:26.841854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a979d44-00b2-46a7-a570-edf5faf77784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:26.843425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a979d44-00b2-46a7-a570-edf5faf77784 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:26.853174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 758/1518] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:35.180442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ceb9-7432-4f41-b254-8e067a24d2b6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:35.180913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ceb9-7432-4f41-b254-8e067a24d2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:35.181050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ceb9-7432-4f41-b254-8e067a24d2b6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:35.182361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 761/1519] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 9 23:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:35.187198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68bca2-4f87-4368-8072-98e78a35601e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:35.187653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68bca2-4f87-4368-8072-98e78a35601e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:35.187777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68bca2-4f87-4368-8072-98e78a35601e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:35.208438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dd68bca2-4f87-4368-8072-98e78a35601e None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:35.209667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dd68bca2-4f87-4368-8072-98e78a35601e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:35.218445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 759/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 9 23:00:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:35.225941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ceb9-7432-4f41-b254-8e067a24d2b6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:35.239234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ceb9-7432-4f41-b254-8e067a24d2b6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:35.244672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbfb8b-0715-480e-a994-667f6ecd86bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:35.245258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbfb8b-0715-480e-a994-667f6ecd86bf service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:35.245538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbfb8b-0715-480e-a994-667f6ecd86bf service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:35.245809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbfb8b-0715-480e-a994-667f6ecd86bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:35.253886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbfb8b-0715-480e-a994-667f6ecd86bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:35.256127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbfb8b-0715-480e-a994-667f6ecd86bf service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:35.265707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 762/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:38.321671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecfeec-eb89-4f66-9be9-8219a221f892 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:38.322257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecfeec-eb89-4f66-9be9-8219a221f892 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:38.322257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecfeec-eb89-4f66-9be9-8219a221f892 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:38.323592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:38.329968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be92ad-514a-4da9-888f-9ea7841488b7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:38.330429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be92ad-514a-4da9-888f-9ea7841488b7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:38.330536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be92ad-514a-4da9-888f-9ea7841488b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:38.350493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-23be92ad-514a-4da9-888f-9ea7841488b7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:38.351724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23be92ad-514a-4da9-888f-9ea7841488b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:38.359617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:38.515293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecfeec-eb89-4f66-9be9-8219a221f892 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:38.531157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecfeec-eb89-4f66-9be9-8219a221f892 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:38.537114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb2358-03c7-4fb4-923a-6c45a615680e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:38.537880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2358-03c7-4fb4-923a-6c45a615680e service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:38.538165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2358-03c7-4fb4-923a-6c45a615680e service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:38.538434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2358-03c7-4fb4-923a-6c45a615680e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:38.548219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb2358-03c7-4fb4-923a-6c45a615680e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:38.549817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb2358-03c7-4fb4-923a-6c45a615680e service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:38.560541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 761/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:39.642524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a5170-e5fe-475f-aa60-893d950dca65 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:39.643121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a5170-e5fe-475f-aa60-893d950dca65 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:39.643121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a5170-e5fe-475f-aa60-893d950dca65 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:39.644424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:39.650591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33dfda-db23-4c57-8641-da2f4b8f2808 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:39.650993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33dfda-db23-4c57-8641-da2f4b8f2808 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:39.651145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33dfda-db23-4c57-8641-da2f4b8f2808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:39.670229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5c33dfda-db23-4c57-8641-da2f4b8f2808 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:39.671427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5c33dfda-db23-4c57-8641-da2f4b8f2808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:39.679962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:39.829744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a5170-e5fe-475f-aa60-893d950dca65 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:39.843723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a5170-e5fe-475f-aa60-893d950dca65 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:39.848993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbf0f8-c104-461a-b914-ff08181bb7a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:39.849601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf0f8-c104-461a-b914-ff08181bb7a9 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:39.849876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf0f8-c104-461a-b914-ff08181bb7a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:39.850143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf0f8-c104-461a-b914-ff08181bb7a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:39.857977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbf0f8-c104-461a-b914-ff08181bb7a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:39.859559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbf0f8-c104-461a-b914-ff08181bb7a9 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:39.869102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 765/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:43.837124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c926a4-8947-4c48-b190-82a902c2e1f5 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:43.837667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c926a4-8947-4c48-b190-82a902c2e1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:43.837667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c926a4-8947-4c48-b190-82a902c2e1f5 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:43.838992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:00:43.844212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f585a-7256-4827-a43d-9acdbf895d4c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:43.844610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f585a-7256-4827-a43d-9acdbf895d4c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:43.844796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f585a-7256-4827-a43d-9acdbf895d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:43.864600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cd3f585a-7256-4827-a43d-9acdbf895d4c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:43.865842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cd3f585a-7256-4827-a43d-9acdbf895d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:43.873822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:44.165983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c926a4-8947-4c48-b190-82a902c2e1f5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:44.167699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-30c926a4-8947-4c48-b190-82a902c2e1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:44.201184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:00:44.245869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c926a4-8947-4c48-b190-82a902c2e1f5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:44.249352 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027d8e2-720b-4c44-a3ec-aeffedd7ca47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:44.249950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d8e2-720b-4c44-a3ec-aeffedd7ca47 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:44.250231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d8e2-720b-4c44-a3ec-aeffedd7ca47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:44.250497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d8e2-720b-4c44-a3ec-aeffedd7ca47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:44.259730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027d8e2-720b-4c44-a3ec-aeffedd7ca47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:44.261145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027d8e2-720b-4c44-a3ec-aeffedd7ca47 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:44.270187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 764/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:46.409650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177d6b-125a-4966-af16-cdae9eae9316 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:46.410258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177d6b-125a-4966-af16-cdae9eae9316 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:46.410258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca177d6b-125a-4966-af16-cdae9eae9316 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:46.411595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:46.417435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5b460-ab9a-4078-8ed2-d74fd765dfa9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:46.417862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5b460-ab9a-4078-8ed2-d74fd765dfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:46.417993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5b460-ab9a-4078-8ed2-d74fd765dfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:46.437912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ace5b460-ab9a-4078-8ed2-d74fd765dfa9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:46.439136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ace5b460-ab9a-4078-8ed2-d74fd765dfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:46.446923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:46.736536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca177d6b-125a-4966-af16-cdae9eae9316 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:46.747124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca177d6b-125a-4966-af16-cdae9eae9316 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:46.750690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0ab4c-9a35-442f-a934-f1a958d91694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:46.751282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ab4c-9a35-442f-a934-f1a958d91694 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:46.751594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ab4c-9a35-442f-a934-f1a958d91694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:46.751866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ab4c-9a35-442f-a934-f1a958d91694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:46.761029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0ab4c-9a35-442f-a934-f1a958d91694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:46.762458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0ab4c-9a35-442f-a934-f1a958d91694 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:46.772262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 768/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:50.377206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e413-7e21-4c8a-ab80-f99e51be284f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:50.377756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e413-7e21-4c8a-ab80-f99e51be284f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:50.377756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e413-7e21-4c8a-ab80-f99e51be284f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:50.379078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:50.384431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b19085-bcb8-478a-aed5-d62f331aee91 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:50.384856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b19085-bcb8-478a-aed5-d62f331aee91 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:50.384965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b19085-bcb8-478a-aed5-d62f331aee91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:50.404897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e9b19085-bcb8-478a-aed5-d62f331aee91 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:50.406102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9b19085-bcb8-478a-aed5-d62f331aee91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:50.414605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:50.722316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605e413-7e21-4c8a-ab80-f99e51be284f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:50.724606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1605e413-7e21-4c8a-ab80-f99e51be284f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:50.804138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605e413-7e21-4c8a-ab80-f99e51be284f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:50.807742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b4516-7bf4-4b4b-a513-ede01e03e113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:50.808346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b4516-7bf4-4b4b-a513-ede01e03e113 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:50.808627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b4516-7bf4-4b4b-a513-ede01e03e113 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:50.808899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b4516-7bf4-4b4b-a513-ede01e03e113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:50.817527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b4516-7bf4-4b4b-a513-ede01e03e113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:50.818979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b4516-7bf4-4b4b-a513-ede01e03e113 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:50.828227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 767/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:00:52.976726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8ff5e-8e27-4b33-a39a-4b4a8aad1e56 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:52.977303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8ff5e-8e27-4b33-a39a-4b4a8aad1e56 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:52.977303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8ff5e-8e27-4b33-a39a-4b4a8aad1e56 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:52.978594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:00:52.984021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516fbb9-a0c7-4b05-816e-71597188e2db None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:52.984464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516fbb9-a0c7-4b05-816e-71597188e2db None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:52.984570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516fbb9-a0c7-4b05-816e-71597188e2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:53.003790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b516fbb9-a0c7-4b05-816e-71597188e2db None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:00:53.004993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b516fbb9-a0c7-4b05-816e-71597188e2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:53.013196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:00:53.314122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8ff5e-8e27-4b33-a39a-4b4a8aad1e56 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:00:53.316041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cfb8ff5e-8e27-4b33-a39a-4b4a8aad1e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:00:53.329733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8ff5e-8e27-4b33-a39a-4b4a8aad1e56 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:00:53.333181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93755973-6fb1-4648-9602-695511c575fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:00:53.333817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93755973-6fb1-4648-9602-695511c575fa service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:00:53.334093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93755973-6fb1-4648-9602-695511c575fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:00:53.334358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93755973-6fb1-4648-9602-695511c575fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:00:53.342915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93755973-6fb1-4648-9602-695511c575fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:00:53.344357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93755973-6fb1-4648-9602-695511c575fa service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:00:53.353475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 771/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:01:02.998993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=63 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:03.349804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=37 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:16.561177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f72c7f-1d98-4ada-89da-8b5ef312c50d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.562233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f72c7f-1d98-4ada-89da-8b5ef312c50d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.562233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f72c7f-1d98-4ada-89da-8b5ef312c50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.584558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af28b-a7fe-4b7b-8bc1-09e888dcab9c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.585720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af28b-a7fe-4b7b-8bc1-09e888dcab9c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.585720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af28b-a7fe-4b7b-8bc1-09e888dcab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.605280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e4f72c7f-1d98-4ada-89da-8b5ef312c50d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.606555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e4f72c7f-1d98-4ada-89da-8b5ef312c50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.618880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c97af28b-a7fe-4b7b-8bc1-09e888dcab9c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.620190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c97af28b-a7fe-4b7b-8bc1-09e888dcab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.624985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 769/1540] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.638761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 772/1541] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.646518 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d67bb-cca7-4082-a4f5-d21b4a7dcf33 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.646839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d67bb-cca7-4082-a4f5-d21b4a7dcf33 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.647071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d67bb-cca7-4082-a4f5-d21b4a7dcf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.670989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51aefa-0f30-475d-9fe8-920bb66ae139 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.671256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51aefa-0f30-475d-9fe8-920bb66ae139 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.671564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51aefa-0f30-475d-9fe8-920bb66ae139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.679379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-703d67bb-cca7-4082-a4f5-d21b4a7dcf33 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.680655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-703d67bb-cca7-4082-a4f5-d21b4a7dcf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.703110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6d51aefa-0f30-475d-9fe8-920bb66ae139 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.704788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d51aefa-0f30-475d-9fe8-920bb66ae139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.708137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 770/1542] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.709196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207e84-0cdd-4a80-9e91-890d0e34de78 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.709473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207e84-0cdd-4a80-9e91-890d0e34de78 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.709846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207e84-0cdd-4a80-9e91-890d0e34de78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.739416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 773/1543] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:16.744472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f4207e84-0cdd-4a80-9e91-890d0e34de78 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.745730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f4207e84-0cdd-4a80-9e91-890d0e34de78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.748237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066e749-2a9f-4523-b137-2f42738fb689 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.748541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066e749-2a9f-4523-b137-2f42738fb689 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.748816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066e749-2a9f-4523-b137-2f42738fb689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.766904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 771/1544] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.767793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207e84-0cdd-4a80-9e91-890d0e34de78 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:16.771075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d066e749-2a9f-4523-b137-2f42738fb689 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.776182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d066e749-2a9f-4523-b137-2f42738fb689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.787608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207e84-0cdd-4a80-9e91-890d0e34de78 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:16.789545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 774/1545] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.792612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4de33-50df-43dc-8685-5e13f4bf7896 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.792612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4de33-50df-43dc-8685-5e13f4bf7896 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.792612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4de33-50df-43dc-8685-5e13f4bf7896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.798951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa30d75-fc54-4c3f-9b42-2ecff7f90dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:16.800299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa30d75-fc54-4c3f-9b42-2ecff7f90dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.800813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa30d75-fc54-4c3f-9b42-2ecff7f90dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.801292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa30d75-fc54-4c3f-9b42-2ecff7f90dfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.802335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa30d75-fc54-4c3f-9b42-2ecff7f90dfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:16.802855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa30d75-fc54-4c3f-9b42-2ecff7f90dfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:16.805541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa30d75-fc54-4c3f-9b42-2ecff7f90dfc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:16.816739 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-05c4de33-50df-43dc-8685-5e13f4bf7896 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.817964 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05c4de33-50df-43dc-8685-5e13f4bf7896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.820682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 772/1546] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:16.822024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198daf8-5a1c-4c23-8313-9dc7c3f5495c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.822308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198daf8-5a1c-4c23-8313-9dc7c3f5495c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.822721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198daf8-5a1c-4c23-8313-9dc7c3f5495c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.834205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 775/1547] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.835153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4de33-50df-43dc-8685-5e13f4bf7896 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:16.845911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1198daf8-5a1c-4c23-8313-9dc7c3f5495c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.847272 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1198daf8-5a1c-4c23-8313-9dc7c3f5495c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.852244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4de33-50df-43dc-8685-5e13f4bf7896 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:16.858374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 773/1548] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.858650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:16.859284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.859608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.859882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.860108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ac5-e6a4-4da8-a95b-860bb2939038 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.860451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ac5-e6a4-4da8-a95b-860bb2939038 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.860571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:16.860736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ac5-e6a4-4da8-a95b-860bb2939038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.860846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:16.862898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:02:16.862898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 09 23:02:16.862898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: Sep 09 23:02:16.862898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb380a8-b4ae-43ca-9024-0fd45ddcd5e9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:16.881362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 776/1549] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:16.882793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb6ded-c4e5-47a8-b5bd-2210a674ebf5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.883087 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb6ded-c4e5-47a8-b5bd-2210a674ebf5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.883364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb6ded-c4e5-47a8-b5bd-2210a674ebf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.890080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-50b15ac5-e6a4-4da8-a95b-860bb2939038 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.891633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-50b15ac5-e6a4-4da8-a95b-860bb2939038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.900596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 774/1550] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.901390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ac5-e6a4-4da8-a95b-860bb2939038 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:16.904985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-51bb6ded-c4e5-47a8-b5bd-2210a674ebf5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.906381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-51bb6ded-c4e5-47a8-b5bd-2210a674ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.911421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ac5-e6a4-4da8-a95b-860bb2939038 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:16.916742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07030a74-a554-47d6-abb9-f3811c3849c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:16.917346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07030a74-a554-47d6-abb9-f3811c3849c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.917625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07030a74-a554-47d6-abb9-f3811c3849c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.921306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 777/1551] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.921473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07030a74-a554-47d6-abb9-f3811c3849c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.921473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07030a74-a554-47d6-abb9-f3811c3849c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:16.921641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d90ba-7462-416b-9389-7944c7d8cca9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.921730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-07030a74-a554-47d6-abb9-f3811c3849c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:02:16.921730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 23:02:16.921730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 23:02:16.921991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d90ba-7462-416b-9389-7944c7d8cca9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.921991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d90ba-7462-416b-9389-7944c7d8cca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.922148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07030a74-a554-47d6-abb9-f3811c3849c0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:16.941795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fd5d90ba-7462-416b-9389-7944c7d8cca9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.943019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fd5d90ba-7462-416b-9389-7944c7d8cca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.953024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 778/1552] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:16.957132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d90ba-7462-416b-9389-7944c7d8cca9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:16.961315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 775/1553] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:16.962582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34487099-df07-4b1a-9251-e48a21b33b45 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.962893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34487099-df07-4b1a-9251-e48a21b33b45 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.963137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34487099-df07-4b1a-9251-e48a21b33b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.965974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d90ba-7462-416b-9389-7944c7d8cca9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:16.983885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-34487099-df07-4b1a-9251-e48a21b33b45 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.984735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-34487099-df07-4b1a-9251-e48a21b33b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:17.024281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.024897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.025179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.025447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.026096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.026402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.028068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.040926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 779/1554] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.041817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.048144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 776/1555] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:17.049416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de548-f2eb-4feb-b690-2f2c0d0be53c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.049698 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de548-f2eb-4feb-b690-2f2c0d0be53c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.050224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de548-f2eb-4feb-b690-2f2c0d0be53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.057181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95191ed5-61ab-4538-9766-d4843f0dc1cd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.061899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.062555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.062783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.063056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.063720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.065640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.070123 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c69de548-f2eb-4feb-b690-2f2c0d0be53c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:17.071329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c69de548-f2eb-4feb-b690-2f2c0d0be53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:17.079299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 777/1556] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:17.080738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfafdc9-9004-42b8-be2d-ad6e006222a8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.081025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfafdc9-9004-42b8-be2d-ad6e006222a8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.081293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfafdc9-9004-42b8-be2d-ad6e006222a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.100424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 780/1557] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.100424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.109850 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dcfafdc9-9004-42b8-be2d-ad6e006222a8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:17.111212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dcfafdc9-9004-42b8-be2d-ad6e006222a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:17.162707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb73e5-552a-4dda-a604-770538526904 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.168050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.168663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.168975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.169217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.169864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.170169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.175709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.175866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 778/1558] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:17.175866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfafdc9-9004-42b8-be2d-ad6e006222a8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.192315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 781/1559] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.193184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfafdc9-9004-42b8-be2d-ad6e006222a8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.193282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.198659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf405b-f083-4b21-835e-ff0d3d4d927e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.199784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.200432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.200875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.200995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.202141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.202447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.203521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38190e59-108f-4d34-9008-6feb0d0364c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.204056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.204157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190e59-108f-4d34-9008-6feb0d0364c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.204392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190e59-108f-4d34-9008-6feb0d0364c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.204653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190e59-108f-4d34-9008-6feb0d0364c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.205282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38190e59-108f-4d34-9008-6feb0d0364c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.206285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-38190e59-108f-4d34-9008-6feb0d0364c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:02:17.206285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 09 23:02:17.206285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: Sep 09 23:02:17.207472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38190e59-108f-4d34-9008-6feb0d0364c0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.216853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 779/1560] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.218150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.224197 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.224356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.230534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d0dac-1d61-484f-aca5-99f8b405b48b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.237120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcb710-2d4f-4991-9653-bbaeb5095f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.237721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcb710-2d4f-4991-9653-bbaeb5095f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.237997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcb710-2d4f-4991-9653-bbaeb5095f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.238266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcb710-2d4f-4991-9653-bbaeb5095f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.238887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcb710-2d4f-4991-9653-bbaeb5095f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.240723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 782/1561] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/users => generated 284 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.240875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcb710-2d4f-4991-9653-bbaeb5095f53 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.242026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17920730-03f9-4242-b7fb-9299fefe1592 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.242320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17920730-03f9-4242-b7fb-9299fefe1592 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.242622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17920730-03f9-4242-b7fb-9299fefe1592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.262109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-17920730-03f9-4242-b7fb-9299fefe1592 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:17.263322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-17920730-03f9-4242-b7fb-9299fefe1592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:17.269534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 780/1562] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.270808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f417a5-e2b8-4d3f-8135-c93719f693d8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.271095 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f417a5-e2b8-4d3f-8135-c93719f693d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.271367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f417a5-e2b8-4d3f-8135-c93719f693d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.274890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 783/1563] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:17.276032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17920730-03f9-4242-b7fb-9299fefe1592 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.285676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17920730-03f9-4242-b7fb-9299fefe1592 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.292106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-90f417a5-e2b8-4d3f-8135-c93719f693d8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:17.293334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-90f417a5-e2b8-4d3f-8135-c93719f693d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:17.343841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.344381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.344659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.344931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.345595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.347521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.363617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 781/1564] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:17.364439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f417a5-e2b8-4d3f-8135-c93719f693d8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.371206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 784/1565] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.372245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.374009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f417a5-e2b8-4d3f-8135-c93719f693d8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.378490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.379079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.379355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.379657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.380292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.380589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.382245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6e629-ba6c-4822-bed2-b6fa12806b60 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.382705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.386764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.387343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.387656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.387923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.388567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.390418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.400727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 782/1566] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.400727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.412432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c550e-a07c-45cd-90fb-69482505c6ee admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.417622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.418212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.418490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.418756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.419376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.420080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 785/1567] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.421013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.424431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.452477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 783/1568] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.453350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.487257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df17a4e-42c2-4d90-a615-3da025237fad admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.514148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d70ba-719e-4096-b53e-25cf0222b47d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.518681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.519280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.519586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.519861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.520497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.523009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.544965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.545576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.545854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.546123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.546842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.548542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.548793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 784/1569] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.549752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.554032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.562153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150fba6-d553-480b-b365-93475ea29861 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.562904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 786/1570] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:17.562904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.566100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.567291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.567291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.567769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.568545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.568998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.571280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.571480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d09df-4516-4bb0-a212-e59f1c39b26b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.576084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.576663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.576937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.577204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.577827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.578123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.579698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.587708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 785/1571] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.588994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.598845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 787/1572] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.600073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.604646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f81cba-12c0-4816-b212-bc4e4d19db8a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.611581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.612497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.612946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.613398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.614460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.617028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156ac68-372b-4fa3-83cd-abbe629b3558 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.617825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.623128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.623749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.624028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.624296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.624927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.626786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.629625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 786/1573] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:17.630199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.640142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89969-f619-49fa-8fa9-42c0776db8bc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.644727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.645297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.645580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.645854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.646503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.648504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.654456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 788/1574] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.654456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.659693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.672384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 787/1575] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.673914 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.685333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d0aff-9371-46df-ba99-5034d7e38bf9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.718341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36195e-699d-47dd-9d2b-219964639b8e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.722925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.723557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.723840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.724107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.724732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.730207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.744201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.744821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.745145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.745404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.746042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.747664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.750204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 789/1576] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.751354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.757510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 788/1577] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:17.758622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.762434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae9153-75fb-47df-a174-834533c2a17e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.767203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.767753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8cea0-37c9-4ed0-8f79-691233bfd1eb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.767874 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.768098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.768373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.769031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.771177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.773201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.773848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.774131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.774398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.775017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.776957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.782506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 790/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:17.783307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.795792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc73cff-48f7-40e2-8378-cc115e97ed7f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.801425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.802048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.802368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.802711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.803369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.805324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.810295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 789/1579] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.810295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.835597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 791/1580] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.835597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.873001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18913369-bf60-4e46-8800-3abf8cb51b35 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.896630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc795a48-fbb9-4d2d-bc00-d85edf5627fc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.901344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.901926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.902208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.902479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.903110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.905079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.931341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 792/1581] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.932300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.942622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4605e2d-1d38-4a68-a4b9-6f099fd09935 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.944809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.945338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.945720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.945996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.946749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.948366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.948541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.949161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.949442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.949710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.950341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.951966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.957202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.962805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 790/1582] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:17.962805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.964405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 793/1583] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:17.965599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.969912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd09e8c-7941-4762-b3d9-1c5a7d01c246 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.974527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.975132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.975445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.975749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.976380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.977278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84911c13-388b-4785-b986-2dedb535fbbd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.977967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.982365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.982958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.983706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.983986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.984749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.986511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.987985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 791/1584] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:17.988751 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.993970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.000318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6651-9c9f-4a1c-be28-35dba86a75fe admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.000743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 794/1585] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.001660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.006236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.007690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.007974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.008727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] PATH_INFO: `/v3/projects/20fcdfc6f40b46adbf6d06c914bd2ffc/users/d2b35881d3824c85badb4c7a67f3262c/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.011864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcdd97-1319-462c-ad3f-dd7edbcda773 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.016721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.017325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.017602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.017869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.018482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.020480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.023204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fcdfc6f40b46adbf6d06c914bd2ffc, user_id=d2b35881d3824c85badb4c7a67f3262c, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.024357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:02:18.024357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 23:02:18.024357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 23:02:18.026284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.041275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 792/1586] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:17 2022] PUT /identity/v3/projects/20fcdfc6f40b46adbf6d06c914bd2ffc/users/d2b35881d3824c85badb4c7a67f3262c/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:18.042089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 795/1587] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:18.042913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.044302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.053955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c568fa-cf5f-4b12-bf92-111a2fe965be admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.108247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e667d4-a0f9-4026-bd72-1b86d581ec3c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.109981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.110596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.110879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.111154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.111893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.113498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.113814 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.114411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.114692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.114957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.115617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.118124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.124431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 796/1588] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.125000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.136905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640a05-8b7f-4bdd-9ccd-c0bf6df6d037 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.141567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 793/1589] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:18.141567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.143579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.144341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.144513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.144790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] PATH_INFO: `/v3/projects/d688ef3e358c4822823707982fe4f654/users/e24e6fb4c69d48f7b2972b53fb8ea4a1/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.148923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafcf6d-928a-43b5-b660-94a44ad19b6c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.153382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.153957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.154623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.154623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.155110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.156719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.160587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d688ef3e358c4822823707982fe4f654, user_id=e24e6fb4c69d48f7b2972b53fb8ea4a1, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.163735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.170107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 794/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.170107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.179657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e97f-537a-4814-ad10-88c5b1f4228d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.180298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 797/1591] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:18 2022] PUT /identity/v3/projects/d688ef3e358c4822823707982fe4f654/users/e24e6fb4c69d48f7b2972b53fb8ea4a1/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:18.181107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.242607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdc504-e943-4de0-adcc-56331d1f258f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.243222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdc504-e943-4de0-adcc-56331d1f258f admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.243542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdc504-e943-4de0-adcc-56331d1f258f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.243803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdc504-e943-4de0-adcc-56331d1f258f admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.244440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdc504-e943-4de0-adcc-56331d1f258f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.246025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdc504-e943-4de0-adcc-56331d1f258f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.247230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec019f1-8c46-40bd-973c-3ae16b1ddc61 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.251884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.252537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.252819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.253092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] PATH_INFO: `/v3/projects/59e9a7accd994fdcb71b007c6ef9be43/users/86667d934a8a4b438764995fedb3feae/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.255730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 795/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.256940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb87b-4889-4f8b-b6c8-b13cf9115a92 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.257230 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb87b-4889-4f8b-b6c8-b13cf9115a92 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.257507 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb87b-4889-4f8b-b6c8-b13cf9115a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.271134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e9a7accd994fdcb71b007c6ef9be43, user_id=86667d934a8a4b438764995fedb3feae, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.273945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.284452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-22edb87b-4889-4f8b-b6c8-b13cf9115a92 None None] MFA Rules not processed for user `d2b35881d3824c85badb4c7a67f3262c`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:18.285673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-22edb87b-4889-4f8b-b6c8-b13cf9115a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:18.287263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 798/1593] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:18 2022] PUT /identity/v3/projects/59e9a7accd994fdcb71b007c6ef9be43/users/86667d934a8a4b438764995fedb3feae/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:18.288600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.324569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.336890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.360997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b378594-46ea-43c2-bd84-a81d9af8dc19 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.361324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 796/1594] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:18.362213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb87b-4889-4f8b-b6c8-b13cf9115a92 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.371367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.372296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.373180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.373180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.374064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.375861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.377005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb87b-4889-4f8b-b6c8-b13cf9115a92 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.381651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.382409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.382688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.382960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.383641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.385245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.386525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 799/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.387138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.396329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81390d04-00c7-4079-88cc-6db9c68befa2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.398864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 797/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.399615 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.400998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d9d72-b77e-4b73-83af-cb5e1860b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.401573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d9d72-b77e-4b73-83af-cb5e1860b992 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.401918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d9d72-b77e-4b73-83af-cb5e1860b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.402109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d9d72-b77e-4b73-83af-cb5e1860b992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.402718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d9d72-b77e-4b73-83af-cb5e1860b992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.404475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d9d72-b77e-4b73-83af-cb5e1860b992 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.409085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d5965-a986-4d66-9e00-a2a6b2acdc42 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.418571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 800/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.418738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ca46-8dec-438f-9139-869c3ebac47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.418738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ca46-8dec-438f-9139-869c3ebac47e admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.419035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1f1be-033d-4005-bde2-b151990dded1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.419133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ca46-8dec-438f-9139-869c3ebac47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.419230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1f1be-033d-4005-bde2-b151990dded1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.419319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ca46-8dec-438f-9139-869c3ebac47e admin admin] PATH_INFO: `/v3/projects/8084c097adde47ffadcacca8f4d3ea73/users/eaef8f72768e4d1daf21257fdaf580d2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.419416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1f1be-033d-4005-bde2-b151990dded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.435046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ca46-8dec-438f-9139-869c3ebac47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8084c097adde47ffadcacca8f4d3ea73, user_id=eaef8f72768e4d1daf21257fdaf580d2, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.436353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7ba1f1be-033d-4005-bde2-b151990dded1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:18.437545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7ba1f1be-033d-4005-bde2-b151990dded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:18.437705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ca46-8dec-438f-9139-869c3ebac47e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.446333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 801/1598] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:18.447083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1f1be-033d-4005-bde2-b151990dded1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.456193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 798/1599] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:18 2022] PUT /identity/v3/projects/8084c097adde47ffadcacca8f4d3ea73/users/eaef8f72768e4d1daf21257fdaf580d2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:18.456440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1f1be-033d-4005-bde2-b151990dded1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.457495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030fda5-9418-4bfa-8bc1-552fa10f4d2f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.457771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030fda5-9418-4bfa-8bc1-552fa10f4d2f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.458038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030fda5-9418-4bfa-8bc1-552fa10f4d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.485450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3030fda5-9418-4bfa-8bc1-552fa10f4d2f None None] MFA Rules not processed for user `86667d934a8a4b438764995fedb3feae`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:18.486646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3030fda5-9418-4bfa-8bc1-552fa10f4d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:18.513461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.514140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.514399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.514694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] PATH_INFO: `/v3/projects/e35b3b47d92b4b0d9be7370466a8d03d/users/97ff9f664ab6462a8d23af92109e6399/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.529341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35b3b47d92b4b0d9be7370466a8d03d, user_id=97ff9f664ab6462a8d23af92109e6399, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.532195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.546671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 802/1600] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:18 2022] PUT /identity/v3/projects/e35b3b47d92b4b0d9be7370466a8d03d/users/97ff9f664ab6462a8d23af92109e6399/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:18.547577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.563096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 799/1601] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:18.567588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030fda5-9418-4bfa-8bc1-552fa10f4d2f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.577455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030fda5-9418-4bfa-8bc1-552fa10f4d2f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.602275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05c82-334b-4e9d-a353-f6c220cadab0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.605924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.606576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.606854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.607122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.634825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.635360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.635663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.635935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.636847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.638442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.648006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 800/1602] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.648843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.649321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.660174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87636836-e022-45cb-91ed-d4c9760e9fea admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.664779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.665422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.665707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.665989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] PATH_INFO: `/v3/projects/6dd5abe437cf4167b87b1525f06fd71d/users/9ca54d4a43b2419f958fe9bbbdbfa94a/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.666941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.668429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.677784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 803/1603] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:18.679094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.681340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd5abe437cf4167b87b1525f06fd71d, user_id=9ca54d4a43b2419f958fe9bbbdbfa94a, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.684023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.690578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709aba-eba2-49ab-baf8-8ec489534061 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.698196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc5ed6-cc10-4317-839a-84dd67245811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.698819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc5ed6-cc10-4317-839a-84dd67245811 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.699151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc5ed6-cc10-4317-839a-84dd67245811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.699701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc5ed6-cc10-4317-839a-84dd67245811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.699836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 801/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:18 2022] PUT /identity/v3/projects/6dd5abe437cf4167b87b1525f06fd71d/users/9ca54d4a43b2419f958fe9bbbdbfa94a/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:18.700107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc5ed6-cc10-4317-839a-84dd67245811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.700437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.701697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc5ed6-cc10-4317-839a-84dd67245811 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.714926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 804/1605] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:18.715209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddd924-b02f-4f51-b041-d5f0777b6933 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.715521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddd924-b02f-4f51-b041-d5f0777b6933 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.715793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddd924-b02f-4f51-b041-d5f0777b6933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.722102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.737161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1fddd924-b02f-4f51-b041-d5f0777b6933 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:18.738442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1fddd924-b02f-4f51-b041-d5f0777b6933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:18.767036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baa217-affd-4309-9582-50b9a5367073 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.771755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.773105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.773392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.773666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.804974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 805/1606] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:18.807424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321dba8-cdb0-4eb7-b4c4-e88d26f31bd8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.808146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321dba8-cdb0-4eb7-b4c4-e88d26f31bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.809162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321dba8-cdb0-4eb7-b4c4-e88d26f31bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.838878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2321dba8-cdb0-4eb7-b4c4-e88d26f31bd8 None None] MFA Rules not processed for user `97ff9f664ab6462a8d23af92109e6399`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:18.840263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2321dba8-cdb0-4eb7-b4c4-e88d26f31bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:18.843817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.845266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.855901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 802/1607] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:18.857616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.868592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fd32-4c94-4f19-bfa2-32a6da39055d service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.873385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.874176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.874462 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.874737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.875374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.877322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.922949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 803/1608] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:18.924219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.924802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 806/1609] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:18.926817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a40c-cd28-42b3-bb71-5926523bcccf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.927203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a40c-cd28-42b3-bb71-5926523bcccf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.927613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a40c-cd28-42b3-bb71-5926523bcccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.961953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bc27a40c-cd28-42b3-bb71-5926523bcccf None None] MFA Rules not processed for user `9ca54d4a43b2419f958fe9bbbdbfa94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:18.964193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bc27a40c-cd28-42b3-bb71-5926523bcccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:19.012010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2346a-3e40-48d4-a5ca-f05cc4d1445a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.019419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.021190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.023949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.023949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] PATH_INFO: `/v3/projects/e966946c8e5a4e71803e8e37971160e1/users/56a554beedec4915b6ad1066823c8d4e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.036369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.051041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e966946c8e5a4e71803e8e37971160e1, user_id=56a554beedec4915b6ad1066823c8d4e, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.056032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.058260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 807/1610] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:19.059298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27a40c-cd28-42b3-bb71-5926523bcccf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.072048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27a40c-cd28-42b3-bb71-5926523bcccf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.076508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 804/1611] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:18 2022] PUT /identity/v3/projects/e966946c8e5a4e71803e8e37971160e1/users/56a554beedec4915b6ad1066823c8d4e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:19.078061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.081374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:19.134274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.139836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.140706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.141071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.141341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] PATH_INFO: `/v3/projects/cba38c443abf49a48bdb55f42dac68b0/users/85cd8a90230b4c69886534be67acd943/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.160691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba38c443abf49a48bdb55f42dac68b0, user_id=85cd8a90230b4c69886534be67acd943, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.163999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.182278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 808/1612] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:19 2022] PUT /identity/v3/projects/cba38c443abf49a48bdb55f42dac68b0/users/85cd8a90230b4c69886534be67acd943/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:19.183565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.198131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71a60-6507-4377-a834-148a2bfe0d2c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.266294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b154-8d6b-46b6-904f-c7fc2781fff4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.273277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.275141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.275417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.276582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.277908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.280110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.284492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.285115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.285399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.285674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.295968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.297445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.310595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 805/1613] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:19.311761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.318557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 809/1614] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:19 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:19.320709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.322875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07352475-ae86-4912-9376-c8da868fce1f service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.330719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fab542-b80b-4674-8bcb-047a3ffb3c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.331344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab542-b80b-4674-8bcb-047a3ffb3c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.331668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab542-b80b-4674-8bcb-047a3ffb3c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.331951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab542-b80b-4674-8bcb-047a3ffb3c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.332649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab542-b80b-4674-8bcb-047a3ffb3c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.334638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab542-b80b-4674-8bcb-047a3ffb3c4a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.335225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf55ea-3a01-4d90-987f-2eb7b3c6bee4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.342370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1dc68-4df2-4933-9781-db49820fd7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.343669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1dc68-4df2-4933-9781-db49820fd7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.344237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1dc68-4df2-4933-9781-db49820fd7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.344858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1dc68-4df2-4933-9781-db49820fd7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.345824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1dc68-4df2-4933-9781-db49820fd7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.349080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1dc68-4df2-4933-9781-db49820fd7f8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.378936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 806/1615] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:19 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:19.382721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909e114-8d16-4681-9bbb-1491730a13bb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.383021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909e114-8d16-4681-9bbb-1491730a13bb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.383309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909e114-8d16-4681-9bbb-1491730a13bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.399889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 810/1616] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:19 2022] POST /identity/v3/projects => generated 363 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:19.401406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e610d7f-0e1a-4014-a7b4-8e058508344b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.409379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e610d7f-0e1a-4014-a7b4-8e058508344b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.409379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e610d7f-0e1a-4014-a7b4-8e058508344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.426560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1909e114-8d16-4681-9bbb-1491730a13bb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:19.429940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1909e114-8d16-4681-9bbb-1491730a13bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:19.449658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3e610d7f-0e1a-4014-a7b4-8e058508344b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:19.451026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3e610d7f-0e1a-4014-a7b4-8e058508344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:19.503873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.521470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 807/1617] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:19.523409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909e114-8d16-4681-9bbb-1491730a13bb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.535080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909e114-8d16-4681-9bbb-1491730a13bb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.541022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.541910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.542344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.542680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.543731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.546034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.546133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 811/1618] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:19.547818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e610d7f-0e1a-4014-a7b4-8e058508344b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.562514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 808/1619] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:19.563073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e610d7f-0e1a-4014-a7b4-8e058508344b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.563895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.571728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.572707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.573049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.573331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.574076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.575181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954e66-bfd9-4efd-bdef-a8d85ee3b9ab admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.576325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.581046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.581801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.582150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.582490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.583367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.586284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.610342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 812/1620] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:19 2022] POST /identity/v3/users => generated 296 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:19.610342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.618165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 809/1621] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:19 2022] POST /identity/v3/users => generated 288 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:19.618165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.656079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.683434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771a4f2-b237-4fb7-be2a-812400585f28 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.684748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e26b8f-6342-4c7c-bffb-bb41ef3fb721 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.687559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.688252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.688605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.688946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.689838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.690332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.690926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.691324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.699903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.701434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.703355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.705074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.715220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 813/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:19.715220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.732528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 810/1623] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:19.733532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.734746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9564-871b-416a-b8fc-85898078904f service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.741119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.742361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.742923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.743416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] PATH_INFO: `/v3/projects/1b19d306684d4e9e80ba5db56585ca1c/users/f1b0625aef934c9dbf928a41cec2301b/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.746758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec55b-18f1-4aac-84b9-51de718c392e service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.763711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.763711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.763711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.763711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.764269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.764269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.783521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b19d306684d4e9e80ba5db56585ca1c, user_id=f1b0625aef934c9dbf928a41cec2301b, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.787175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 811/1624] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:19.787175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.791609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.802970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ea93e-2ff8-4397-a70e-b927cb35945a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.814804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 814/1625] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:19 2022] PUT /identity/v3/projects/1b19d306684d4e9e80ba5db56585ca1c/users/f1b0625aef934c9dbf928a41cec2301b/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:19.814804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.877873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.878607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.878897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.879212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.879944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.882225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.898997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 812/1626] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:19.900044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.902641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f6962-d7d4-44fe-96e0-cd1e61f2cbe6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.910069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.913092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.913412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.914488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] PATH_INFO: `/v3/projects/f913b0ac8aff4e3d95d13ef04304b786/users/ca63510b53774ffab1cb46d0c93125b8/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.914694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837f38e-2fc5-4441-9eb9-f16c4c991c83 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.920308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.920934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.921637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.921975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.923640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.925531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.942504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 813/1627] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:19.943623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.955850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f913b0ac8aff4e3d95d13ef04304b786, user_id=ca63510b53774ffab1cb46d0c93125b8, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.956076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c3d0d-4cee-4736-b53c-eea2d0fd8107 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.962075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.962262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.963141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.963435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.963748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] PATH_INFO: `/v3/projects/0aede3414ef249acbc51938330ea2a53/users/c6c8a82d92b04ac5ac2e45e9aa0c3370/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.990160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aede3414ef249acbc51938330ea2a53, user_id=c6c8a82d92b04ac5ac2e45e9aa0c3370, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.990469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 815/1628] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:19 2022] PUT /identity/v3/projects/f913b0ac8aff4e3d95d13ef04304b786/users/ca63510b53774ffab1cb46d0c93125b8/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:19.994178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.996564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.014417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 814/1629] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:19 2022] PUT /identity/v3/projects/0aede3414ef249acbc51938330ea2a53/users/c6c8a82d92b04ac5ac2e45e9aa0c3370/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:20.015690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.059903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.116532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5998-eca0-4674-b3a6-af6de770d5fd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.118604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d1013-8dbb-451a-8a1c-feed089f68cb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.122129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.122724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.123005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.123279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.124209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.126097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.131473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.132759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.133040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.133313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] PATH_INFO: `/v3/projects/1b19d306684d4e9e80ba5db56585ca1c/users/f1b0625aef934c9dbf928a41cec2301b/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.138069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b19d306684d4e9e80ba5db56585ca1c, user_id=f1b0625aef934c9dbf928a41cec2301b, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.139674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 815/1630] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:20.140506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.144132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.158881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0e7ed-a3b8-4737-b05a-54809f8534dc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.160556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 816/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:19 2022] PUT /identity/v3/projects/1b19d306684d4e9e80ba5db56585ca1c/users/f1b0625aef934c9dbf928a41cec2301b/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:20.170087 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.185478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.266512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d1eff-1a93-4a64-889d-7cb62e1e7463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.267141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1eff-1a93-4a64-889d-7cb62e1e7463 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.267418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1eff-1a93-4a64-889d-7cb62e1e7463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.267725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1eff-1a93-4a64-889d-7cb62e1e7463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.268635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d1eff-1a93-4a64-889d-7cb62e1e7463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.270590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d1eff-1a93-4a64-889d-7cb62e1e7463 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.278085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042e08b-782c-4abf-b05c-7bf9d966359f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.286487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 816/1632] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:20.288881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f6aa6-c042-40dd-87aa-1693d7da6ae0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.288881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f6aa6-c042-40dd-87aa-1693d7da6ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.289207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f6aa6-c042-40dd-87aa-1693d7da6ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.295913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.296641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.296923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.297261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] PATH_INFO: `/v3/projects/f913b0ac8aff4e3d95d13ef04304b786/users/ca63510b53774ffab1cb46d0c93125b8/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.302501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f913b0ac8aff4e3d95d13ef04304b786, user_id=ca63510b53774ffab1cb46d0c93125b8, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.310286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.324089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 817/1633] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:20 2022] PUT /identity/v3/projects/f913b0ac8aff4e3d95d13ef04304b786/users/ca63510b53774ffab1cb46d0c93125b8/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:20.325087 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.347201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2d1f6aa6-c042-40dd-87aa-1693d7da6ae0 None None] MFA Rules not processed for user `f1b0625aef934c9dbf928a41cec2301b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:20.348586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2d1f6aa6-c042-40dd-87aa-1693d7da6ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:20.479793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffebfc5-cd2b-483b-b716-bac353dac4e6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.493656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.494325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.494671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.494882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] PATH_INFO: `/v3/projects/0aede3414ef249acbc51938330ea2a53/users/c6c8a82d92b04ac5ac2e45e9aa0c3370/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.496563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 817/1634] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:20.498123 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e08b7-40d1-4519-9e7a-a8fb052abbf0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.498430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e08b7-40d1-4519-9e7a-a8fb052abbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.498678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e08b7-40d1-4519-9e7a-a8fb052abbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.514855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aede3414ef249acbc51938330ea2a53, user_id=c6c8a82d92b04ac5ac2e45e9aa0c3370, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.517966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.542586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 818/1635] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:20 2022] PUT /identity/v3/projects/0aede3414ef249acbc51938330ea2a53/users/c6c8a82d92b04ac5ac2e45e9aa0c3370/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:20.543568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.549439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-653e08b7-40d1-4519-9e7a-a8fb052abbf0 None None] MFA Rules not processed for user `ca63510b53774ffab1cb46d0c93125b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:20.550977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-653e08b7-40d1-4519-9e7a-a8fb052abbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:20.650322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.657875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108b520-25ad-462c-9ca0-722b79f6dcd8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.666650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.667190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.667464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.667755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.698524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 818/1636] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:20.700810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95477bf5-f8bb-4a60-b45b-866a2aa5b007 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.701097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95477bf5-f8bb-4a60-b45b-866a2aa5b007 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.701423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95477bf5-f8bb-4a60-b45b-866a2aa5b007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.780916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-95477bf5-f8bb-4a60-b45b-866a2aa5b007 None None] MFA Rules not processed for user `c6c8a82d92b04ac5ac2e45e9aa0c3370`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:20.783659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-95477bf5-f8bb-4a60-b45b-866a2aa5b007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:20.816767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.827636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.832561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.863712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 819/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:20.869551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.895418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1d929-3ab1-442e-acf2-340a7fd99aed service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.905302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.906410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.908358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.908678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.958878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.959849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.980775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 819/1638] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:20.988461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 820/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:21.081060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.091256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1580-d371-4ea6-9993-98177c2ed2bb service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:21.095045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:21.096198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.096482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.096888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.117139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:21.117139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:21.138690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f5852-fce9-424e-b8e1-ec9f1424961f None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.139020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f5852-fce9-424e-b8e1-ec9f1424961f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.139242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f5852-fce9-424e-b8e1-ec9f1424961f None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.148724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 820/1640] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 23:02:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:21.153006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 821/1641] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:02:21.160733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002645-9701-40b6-b0e1-f17255d56c2a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.161040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002645-9701-40b6-b0e1-f17255d56c2a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.161314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002645-9701-40b6-b0e1-f17255d56c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.175910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.209720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826775e-abe7-4420-9763-549a7915153a service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:21.229543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790b711-0f7f-4fa1-a7df-a5d1f0071451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:21.231822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b711-0f7f-4fa1-a7df-a5d1f0071451 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.233009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b711-0f7f-4fa1-a7df-a5d1f0071451 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.234353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790b711-0f7f-4fa1-a7df-a5d1f0071451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.298686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-12002645-9701-40b6-b0e1-f17255d56c2a None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:21.306994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-12002645-9701-40b6-b0e1-f17255d56c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:21.433929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790b711-0f7f-4fa1-a7df-a5d1f0071451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:21.438106 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790b711-0f7f-4fa1-a7df-a5d1f0071451 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:21.463774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 822/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:21.490137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 821/1643] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:21.497039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a184-313e-430a-b581-3f26a2f3e0c5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.497829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a184-313e-430a-b581-3f26a2f3e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.498906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a184-313e-430a-b581-3f26a2f3e0c5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.501914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 823/1644] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 23:02:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:21.509695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9871c-4d89-4ca8-a75c-4e1cf7b69e3b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.510358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9871c-4d89-4ca8-a75c-4e1cf7b69e3b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.512150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9871c-4d89-4ca8-a75c-4e1cf7b69e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.581087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-30d9871c-4d89-4ca8-a75c-4e1cf7b69e3b None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:21.582529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-30d9871c-4d89-4ca8-a75c-4e1cf7b69e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:21.608986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 822/1645] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:21.788760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937a184-313e-430a-b581-3f26a2f3e0c5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.803327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937a184-313e-430a-b581-3f26a2f3e0c5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:21.809005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:21.810865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.811503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.811956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.828940 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:21.831271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:21.864605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 824/1646] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:22.031045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9871c-4d89-4ca8-a75c-4e1cf7b69e3b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.048593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9871c-4d89-4ca8-a75c-4e1cf7b69e3b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.065700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca63510b53774ffab1cb46d0c93125b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f913b0ac8aff4e3d95d13ef04304b786', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.069679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.070023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.070958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.074689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:22.075048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.080984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.130699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 823/1647] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:22.135236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.157522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071f390-5189-4c7d-b201-e6ff7be273e4 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.172313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca63510b53774ffab1cb46d0c93125b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f913b0ac8aff4e3d95d13ef04304b786', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.172936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.173221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.173497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.174407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.176567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.256045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 825/1648] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:22 2022] POST /identity/v3/projects => generated 354 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:22.256852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.270114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a79fbc-4b4b-4cec-b2d3-5f4fd157331d tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.423755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca63510b53774ffab1cb46d0c93125b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f913b0ac8aff4e3d95d13ef04304b786', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.424456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.424829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.425453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.426109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:22.428569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.430513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.465204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 824/1649] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:22.564153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.577932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfbff-f576-4b10-909b-3477b52cfc51 tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.591510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca63510b53774ffab1cb46d0c93125b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f913b0ac8aff4e3d95d13ef04304b786', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.592240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.592765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.593043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] PATH_INFO: `/v3/projects/184ae2a0724244d5baf416fce015985b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.597148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorizing `identity:delete_project(project_id=184ae2a0724244d5baf416fce015985b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.598204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:02:22.598204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 09 23:02:22.598204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: Sep 09 23:02:22.599410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.738083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:22.795993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '184ae2a0724244d5baf416fce015985b', 'request_id': 'req-ac880848-fc6d-427e-9c88-ce184c419bea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:22.827867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '184ae2a0724244d5baf416fce015985b', 'request_id': 'req-ac880848-fc6d-427e-9c88-ce184c419bea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:22.836606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '184ae2a0724244d5baf416fce015985b', 'request_id': 'req-ac880848-fc6d-427e-9c88-ce184c419bea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:22.861318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] The token cache is being invalidate because project 184ae2a0724244d5baf416fce015985b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:22.861671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:22.862934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:22.869445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 826/1650] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:22 2022] DELETE /identity/v3/projects/184ae2a0724244d5baf416fce015985b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:22.901193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.903227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:22.910345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:23.075005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf8dc-af2a-468c-a559-3b3ec18893fd tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:23.270342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:23.270342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.270342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.270342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] PATH_INFO: `/v3/users/9ca54d4a43b2419f958fe9bbbdbfa94a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.274350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca54d4a43b2419f958fe9bbbdbfa94a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:23.274350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: WARNING py.warnings [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:02:23.274350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: warnings.warn(msg) Sep 09 23:02:23.274350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: Sep 09 23:02:23.274350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:23.361972 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:23.363459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:23.382943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac880848-fc6d-427e-9c88-ce184c419bea tempest-VolumeQuotaClassesTest-812507108 tempest-VolumeQuotaClassesTest-812507108-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:23.425482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:23.425785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.430294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.430576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] PATH_INFO: `/v3/users/97ff9f664ab6462a8d23af92109e6399` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.434878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ff9f664ab6462a8d23af92109e6399)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:23.441196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:23.482826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca54d4a43b2419f958fe9bbbdbfa94a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.484391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca54d4a43b2419f958fe9bbbdbfa94a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.517419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca54d4a43b2419f958fe9bbbdbfa94a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.605944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca54d4a43b2419f958fe9bbbdbfa94a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.607329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8d04dbf5-bd4e-4ee5-a08a-34952f98c6ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca54d4a43b2419f958fe9bbbdbfa94a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.613004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ff9f664ab6462a8d23af92109e6399'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.636000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ff9f664ab6462a8d23af92109e6399'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.641691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 825/1651] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:22 2022] DELETE /identity/v3/users/9ca54d4a43b2419f958fe9bbbdbfa94a => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:23.646529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.647047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.649351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.691860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ff9f664ab6462a8d23af92109e6399'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.695052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ff9f664ab6462a8d23af92109e6399'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.697449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-505b47da-7920-4001-8f8e-8970fe823b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ff9f664ab6462a8d23af92109e6399'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:23.751301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:23.752813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 827/1652] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:23 2022] DELETE /identity/v3/users/97ff9f664ab6462a8d23af92109e6399 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:23.754150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:23.761285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.761626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.762283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.824478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:23.825872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:23.845496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:23.948120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 826/1653] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:24.065378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 828/1654] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:24.088581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:24.092891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:24.095003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:24.097704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:24.235931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6217df-b287-40a0-a33f-6ef2da960883 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:24.244272 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698d51-a33a-4186-9078-5ea755a2efef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:24.246985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698d51-a33a-4186-9078-5ea755a2efef service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.247819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698d51-a33a-4186-9078-5ea755a2efef service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.248301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698d51-a33a-4186-9078-5ea755a2efef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.249552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:24.281679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad698d51-a33a-4186-9078-5ea755a2efef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:24.283831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad698d51-a33a-4186-9078-5ea755a2efef service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:24.303684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 827/1655] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:24.328981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d88c-9c52-4904-bdf0-ce00b71980c0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:24.342480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc2f13-8e58-41ba-9997-fcf00d195559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:24.343358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc2f13-8e58-41ba-9997-fcf00d195559 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.343928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc2f13-8e58-41ba-9997-fcf00d195559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.344527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc2f13-8e58-41ba-9997-fcf00d195559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.349236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acb411-3818-4c02-b08f-77566320b8d4 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.349715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acb411-3818-4c02-b08f-77566320b8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.350143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acb411-3818-4c02-b08f-77566320b8d4 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.352236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 828/1656] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 23:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:24.362684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479c9c3-4296-4313-b456-a8cea29d2e17 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.363333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479c9c3-4296-4313-b456-a8cea29d2e17 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.363782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479c9c3-4296-4313-b456-a8cea29d2e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.375177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc2f13-8e58-41ba-9997-fcf00d195559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:24.377182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc2f13-8e58-41ba-9997-fcf00d195559 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:24.408679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4479c9c3-4296-4313-b456-a8cea29d2e17 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:24.410142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4479c9c3-4296-4313-b456-a8cea29d2e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:24.418186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 829/1657] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:02:24.526786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b407f9c-1ae2-4191-858c-556e49427afe None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.527096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b407f9c-1ae2-4191-858c-556e49427afe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.527389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b407f9c-1ae2-4191-858c-556e49427afe None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.528511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 829/1658] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:24.529181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 830/1659] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 23:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:24.536328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.536924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.537274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.568198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:24.570112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:24.590040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 830/1660] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:24.598181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b407f9c-1ae2-4191-858c-556e49427afe None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:24.601038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b407f9c-1ae2-4191-858c-556e49427afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:24.688978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b407f9c-1ae2-4191-858c-556e49427afe None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:24.700240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:24.701045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.701333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.701611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.715140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:24.716740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:24.790936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 831/1661] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:02:25.351881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.354752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:25.389053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b7825-4ea8-447c-89b6-ff0354b58784 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:25.405728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:25.406501 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.406785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.409029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] PATH_INFO: `/v3/projects/6dd5abe437cf4167b87b1525f06fd71d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.413965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd5abe437cf4167b87b1525f06fd71d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:25.419745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:25.640163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd5abe437cf4167b87b1525f06fd71d', 'request_id': 'req-915fdf38-da2a-4b43-ba80-0568ac0e95dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:25.651096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:25.670018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd5abe437cf4167b87b1525f06fd71d', 'request_id': 'req-915fdf38-da2a-4b43-ba80-0568ac0e95dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:25.700951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd5abe437cf4167b87b1525f06fd71d', 'request_id': 'req-915fdf38-da2a-4b43-ba80-0568ac0e95dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:25.729942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] The token cache is being invalidate because project 6dd5abe437cf4167b87b1525f06fd71d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:25.733543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:25.738609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:25.742368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 831/1662] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:25 2022] DELETE /identity/v3/projects/6dd5abe437cf4167b87b1525f06fd71d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:25.746970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.765097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ba648-fd17-40b8-91f0-91a5f89a27ae service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:25.916460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.917260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:26.024499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.026781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.026781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.027767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] PATH_INFO: `/v3/users/c6c8a82d92b04ac5ac2e45e9aa0c3370` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.031654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c8a82d92b04ac5ac2e45e9aa0c3370)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.037490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.149942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fdf38-da2a-4b43-ba80-0568ac0e95dc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.165955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.167479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.170019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.170338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] PATH_INFO: `/v3/users/86667d934a8a4b438764995fedb3feae` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.172559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86667d934a8a4b438764995fedb3feae)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.176067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.179889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c8a82d92b04ac5ac2e45e9aa0c3370'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.197742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c8a82d92b04ac5ac2e45e9aa0c3370'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.237360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c8a82d92b04ac5ac2e45e9aa0c3370'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.239591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c8a82d92b04ac5ac2e45e9aa0c3370'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.242283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c8a82d92b04ac5ac2e45e9aa0c3370'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.249972 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:26.282677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 832/1663] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:25 2022] DELETE /identity/v3/users/c6c8a82d92b04ac5ac2e45e9aa0c3370 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:26.285277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.291126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:26.348324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86667d934a8a4b438764995fedb3feae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.351201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86667d934a8a4b438764995fedb3feae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.379948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86667d934a8a4b438764995fedb3feae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.423197 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86667d934a8a4b438764995fedb3feae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.423980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2a23a57-449a-40e3-bf42-452dab5e8c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86667d934a8a4b438764995fedb3feae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.455458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 832/1664] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:25 2022] DELETE /identity/v3/users/86667d934a8a4b438764995fedb3feae => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:26.462134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.462422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.462696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.501385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed3091-2fbd-49e9-9aab-487b58e9227b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.537858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:26.540700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:26.641976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.642755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.643176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.643939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] PATH_INFO: `/v3/projects/e35b3b47d92b4b0d9be7370466a8d03d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.645353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35b3b47d92b4b0d9be7370466a8d03d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.648994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.746686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35b3b47d92b4b0d9be7370466a8d03d', 'request_id': 'req-8507ed3a-af43-4513-a32f-1143909233a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.761158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35b3b47d92b4b0d9be7370466a8d03d', 'request_id': 'req-8507ed3a-af43-4513-a32f-1143909233a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.771710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35b3b47d92b4b0d9be7370466a8d03d', 'request_id': 'req-8507ed3a-af43-4513-a32f-1143909233a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.791175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] The token cache is being invalidate because project e35b3b47d92b4b0d9be7370466a8d03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:26.791571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.793213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:26.801731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:26.816675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 833/1665] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:26.822876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 833/1666] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:26 2022] DELETE /identity/v3/projects/e35b3b47d92b4b0d9be7370466a8d03d => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:26.827061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.828443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:26.853053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ef2e1-e5b7-4def-a80b-9006e4c92ac7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.872085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.873504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:26.949612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:26.991169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.991855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.992133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.992404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] PATH_INFO: `/v3/users/ca63510b53774ffab1cb46d0c93125b8` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.994908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca63510b53774ffab1cb46d0c93125b8)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.996963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:27.006541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ed3a-af43-4513-a32f-1143909233a2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:27.026202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:27.026850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:27.027129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:27.027401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:27.051989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:27.053499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:27.090082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 834/1667] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:02:27.134482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca63510b53774ffab1cb46d0c93125b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:27.139676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca63510b53774ffab1cb46d0c93125b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:27.160949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca63510b53774ffab1cb46d0c93125b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:27.198898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca63510b53774ffab1cb46d0c93125b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:27.199661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca63510b53774ffab1cb46d0c93125b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:27.225884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 834/1668] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:26 2022] DELETE /identity/v3/users/ca63510b53774ffab1cb46d0c93125b8 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:29.355843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.357471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:29.377348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb3b6b-55f2-4d0c-aeeb-b987b73e5297 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.507373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.509217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.509917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.510417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] PATH_INFO: `/v3/projects/59e9a7accd994fdcb71b007c6ef9be43` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.512210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e9a7accd994fdcb71b007c6ef9be43)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.515230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.662508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e9a7accd994fdcb71b007c6ef9be43', 'request_id': 'req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:29.672222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e9a7accd994fdcb71b007c6ef9be43', 'request_id': 'req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:29.684601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e9a7accd994fdcb71b007c6ef9be43', 'request_id': 'req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:29.694202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.695814 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:29.717723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] The token cache is being invalidate because project 59e9a7accd994fdcb71b007c6ef9be43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:29.718087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf356823-9f6f-478b-b09a-02c6391f8430 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.718639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:29.720737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:29.723025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 835/1669] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:29 2022] DELETE /identity/v3/projects/59e9a7accd994fdcb71b007c6ef9be43 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:29.731939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.731939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:29.846483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.847463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.847984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.848458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] PATH_INFO: `/v3/projects/f913b0ac8aff4e3d95d13ef04304b786` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.851432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f913b0ac8aff4e3d95d13ef04304b786)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.854261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.881464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90498e4c-b8a6-485e-8ac1-7e9ed71844fa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.886362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.888111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.889264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.889264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] PATH_INFO: `/v3/users/f1b0625aef934c9dbf928a41cec2301b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.891074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b0625aef934c9dbf928a41cec2301b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.894347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.903872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:29.915917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:29.978357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f913b0ac8aff4e3d95d13ef04304b786', 'request_id': 'req-fb663403-0f49-4b16-a960-69013a725679'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:29.995863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f913b0ac8aff4e3d95d13ef04304b786', 'request_id': 'req-fb663403-0f49-4b16-a960-69013a725679'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.012080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b0625aef934c9dbf928a41cec2301b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.018212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f913b0ac8aff4e3d95d13ef04304b786', 'request_id': 'req-fb663403-0f49-4b16-a960-69013a725679'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.032483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b0625aef934c9dbf928a41cec2301b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.048643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] The token cache is being invalidate because project f913b0ac8aff4e3d95d13ef04304b786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:30.049027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.051303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.054534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 835/1670] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:29 2022] DELETE /identity/v3/projects/f913b0ac8aff4e3d95d13ef04304b786 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:30.055075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:30.058891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:30.066141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b0625aef934c9dbf928a41cec2301b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.067398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b0625aef934c9dbf928a41cec2301b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.068308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c77daa5-fdef-4732-8026-16bc7a7b91b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b0625aef934c9dbf928a41cec2301b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.135033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 836/1671] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:29 2022] DELETE /identity/v3/users/f1b0625aef934c9dbf928a41cec2301b => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:30.141035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe05be-7618-43c7-9878-164f970e7f3d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.141325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe05be-7618-43c7-9878-164f970e7f3d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.141593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe05be-7618-43c7-9878-164f970e7f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.191042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3dfe05be-7618-43c7-9878-164f970e7f3d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:30.196957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3dfe05be-7618-43c7-9878-164f970e7f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:30.212388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663403-0f49-4b16-a960-69013a725679 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:30.356397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:30.357185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.357521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.357891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] PATH_INFO: `/v3/projects/0aede3414ef249acbc51938330ea2a53` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.360891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aede3414ef249acbc51938330ea2a53)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:30.363155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:30.427657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 837/1672] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:30.451812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1f7c-8351-45a4-98eb-85c08ce1a310 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.452111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1f7c-8351-45a4-98eb-85c08ce1a310 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.452573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1f7c-8351-45a4-98eb-85c08ce1a310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.470866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aede3414ef249acbc51938330ea2a53', 'request_id': 'req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.481012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aede3414ef249acbc51938330ea2a53', 'request_id': 'req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.487916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:30.500062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aede3414ef249acbc51938330ea2a53', 'request_id': 'req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.506433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f6ea1f7c-8351-45a4-98eb-85c08ce1a310 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:30.511324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f6ea1f7c-8351-45a4-98eb-85c08ce1a310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:30.522738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] The token cache is being invalidate because project 0aede3414ef249acbc51938330ea2a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:30.523130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.524497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6f4d6426-1ed5-4a86-8e3a-c8f21dcbb9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:30.530849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 836/1673] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:30 2022] DELETE /identity/v3/projects/0aede3414ef249acbc51938330ea2a53 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:30.536305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c86e-4fd2-46d2-9fda-fa03740cca0f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.536670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c86e-4fd2-46d2-9fda-fa03740cca0f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.536872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c86e-4fd2-46d2-9fda-fa03740cca0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.579392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4851c86e-4fd2-46d2-9fda-fa03740cca0f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:30.581233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4851c86e-4fd2-46d2-9fda-fa03740cca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:30.686937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 838/1674] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:30.691593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1f7c-8351-45a4-98eb-85c08ce1a310 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:30.716171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1f7c-8351-45a4-98eb-85c08ce1a310 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:30.737559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:30.737559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.737559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.737559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.742137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:30.742137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:30.742137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:30.744186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 837/1675] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:30.746289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c005b7-4920-4d44-8b4e-510076a1421e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.746579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c005b7-4920-4d44-8b4e-510076a1421e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.746912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c005b7-4920-4d44-8b4e-510076a1421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.787378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 839/1676] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:30.790698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:30.799912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-44c005b7-4920-4d44-8b4e-510076a1421e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:30.802173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-44c005b7-4920-4d44-8b4e-510076a1421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:30.817312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98710036-c68e-4b31-8ea2-07fe437aad77 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:30.825208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:30.826341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.827050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.827259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 838/1677] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:30.828045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.828201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c005b7-4920-4d44-8b4e-510076a1421e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:30.829373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:30.832517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:30.856140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c005b7-4920-4d44-8b4e-510076a1421e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:30.862169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:30.866755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:30.867621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:30.868039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:30.868477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:30.869271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:30.871718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:30.878717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:30.898426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 840/1678] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:30 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:02:30.902594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:30.914955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 839/1679] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:30.916098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:30.938977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2c2be-e386-4b42-a57f-c2fe854af26e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.060682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58d62e-d8b5-4ed5-b5fb-b1f639ccee08 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.074007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.074846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.075974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.076265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.077734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.080665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.081484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.081802 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.082012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.082826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.083505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.085595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.118851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 841/1680] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:30 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:31.123347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.152976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fcb4e-2f68-4c69-900a-415f8dca8d19 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.156558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 840/1681] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:30 2022] POST /identity/v3/projects => generated 357 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:31.162720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.280989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:31.297006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008097a9-455c-4c00-91e2-b984707d8e81 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.309476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.310273 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.310703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.311113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.312465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.319737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.322145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.324021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.324474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.325290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.327149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.330241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.360256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 842/1682] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:02:31.361674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.370691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 841/1683] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:31 2022] POST /identity/v3/users => generated 279 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:31.371835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.383817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b751-25a9-48cc-b2a1-0f5f1e3cb335 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.392566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.393195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.393497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.393747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.394395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.396247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.397745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f1b76-070b-4b6d-a223-bcdb28ba708f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.412451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 843/1684] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:02:31.413854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.414577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.415424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.415960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.416432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.420108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.422044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.424434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1683-22ca-4904-8375-857a7ce7a881 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.433500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee6e5a-1ffe-4f03-abcb-95690d0df008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.434180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6e5a-1ffe-4f03-abcb-95690d0df008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.436024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6e5a-1ffe-4f03-abcb-95690d0df008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.437216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6e5a-1ffe-4f03-abcb-95690d0df008 admin admin] PATH_INFO: `/v3/projects/09300f5207f046658540d53cdde2b9dd/users/71a11a30b6e84952ace83f2b7785a643/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.452002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 842/1685] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:31.453452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.471793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee6e5a-1ffe-4f03-abcb-95690d0df008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09300f5207f046658540d53cdde2b9dd, user_id=71a11a30b6e84952ace83f2b7785a643, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.474628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b00fb5-9d8e-4246-a52b-c70ee4c52e60 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.482336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee6e5a-1ffe-4f03-abcb-95690d0df008 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.490245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.491624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.491986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.492334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.493314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.498711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.523692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 843/1686] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:31.527547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.538745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 844/1687] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:31 2022] PUT /identity/v3/projects/09300f5207f046658540d53cdde2b9dd/users/71a11a30b6e84952ace83f2b7785a643/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:31.540674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3471b0-d9fe-457b-951a-82ebf63c6723 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.540674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3471b0-d9fe-457b-951a-82ebf63c6723 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.541943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3471b0-d9fe-457b-951a-82ebf63c6723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.549984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ba47d-0116-4cb3-bf2b-f7b2b20fa79e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.610926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5c3471b0-d9fe-457b-951a-82ebf63c6723 None None] MFA Rules not processed for user `71a11a30b6e84952ace83f2b7785a643`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:31.614962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c3471b0-d9fe-457b-951a-82ebf63c6723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:31.708172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:31.747204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47850dc6-9e97-4a4f-a044-436dac9f5b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:31.747935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47850dc6-9e97-4a4f-a044-436dac9f5b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.748224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47850dc6-9e97-4a4f-a044-436dac9f5b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.748505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47850dc6-9e97-4a4f-a044-436dac9f5b31 admin admin] PATH_INFO: `/v3/projects/9d119896b1184c82866680785604bb21/users/fe043673440148389343579de277473d/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.810865 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47850dc6-9e97-4a4f-a044-436dac9f5b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d119896b1184c82866680785604bb21, user_id=fe043673440148389343579de277473d, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:31.813760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47850dc6-9e97-4a4f-a044-436dac9f5b31 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:31.817688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 845/1688] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:31.821454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3471b0-d9fe-457b-951a-82ebf63c6723 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:31.847265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 844/1689] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:31 2022] PUT /identity/v3/projects/9d119896b1184c82866680785604bb21/users/fe043673440148389343579de277473d/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:31.850935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3471b0-d9fe-457b-951a-82ebf63c6723 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:31.858321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42b4c-17d1-4ec1-8d6a-98153f97668e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:31.859251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42b4c-17d1-4ec1-8d6a-98153f97668e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:31.859788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c42b4c-17d1-4ec1-8d6a-98153f97668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:31.970296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b1c42b4c-17d1-4ec1-8d6a-98153f97668e None None] MFA Rules not processed for user `fe043673440148389343579de277473d`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:31.978830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b1c42b4c-17d1-4ec1-8d6a-98153f97668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:32.036433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.038527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.039072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.039413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.040331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.043769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.100928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 846/1690] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:31 2022] POST /identity/v3/projects => generated 379 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:32.102821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.154695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 845/1691] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:32.155907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42b4c-17d1-4ec1-8d6a-98153f97668e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.179700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c42b4c-17d1-4ec1-8d6a-98153f97668e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.240004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634e63a-1012-4fa2-a348-2f7b46dd3412 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.246727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.248020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.248510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.248929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.249983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.251563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:32.253452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.292991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.292991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.292991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.292991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.293425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.294350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 847/1692] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:32.294843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.295806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.309137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e562a5-d8ac-4c70-90c8-c8b98faccbfc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.316284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.317168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.317599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.318017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.318847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.321159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.340517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 846/1693] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/projects => generated 355 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:32.344791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.347368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 848/1694] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:32.348806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.361470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b505469-5c75-44ee-b9fa-2a31aa57c965 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.403975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:32.435625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.436265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.436615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.436779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.437703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.440833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.459376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 849/1695] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:32.461245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.470937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ba0f7-9bed-4e8a-9a12-c01a480f290b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.477760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.478377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.480340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c20ed-a5d6-4f83-975f-32e12d90a89b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.486767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3a53d-3598-4356-88fa-fb0c25af20a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.487469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a53d-3598-4356-88fa-fb0c25af20a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.487824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a53d-3598-4356-88fa-fb0c25af20a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.488100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a53d-3598-4356-88fa-fb0c25af20a3 admin admin] PATH_INFO: `/v3/projects/ec8293c86d394e188ff2c8c88b2d491a/users/d1603025e10b44288d932fb1c0969e72/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.488220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.488454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.489344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.491961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.511351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3a53d-3598-4356-88fa-fb0c25af20a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8293c86d394e188ff2c8c88b2d491a, user_id=d1603025e10b44288d932fb1c0969e72, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.514764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3a53d-3598-4356-88fa-fb0c25af20a3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.527581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 847/1696] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:32.530006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.553729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 850/1697] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:32 2022] PUT /identity/v3/projects/ec8293c86d394e188ff2c8c88b2d491a/users/d1603025e10b44288d932fb1c0969e72/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:32.570368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4f305-ba79-47db-b48f-91966b1cdcc6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.571085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4f305-ba79-47db-b48f-91966b1cdcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.572072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4f305-ba79-47db-b48f-91966b1cdcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.578310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d719e1-eb9f-449a-bd6b-9a4f1fec82f5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.626603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-32c4f305-ba79-47db-b48f-91966b1cdcc6 None None] MFA Rules not processed for user `d1603025e10b44288d932fb1c0969e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:32.635224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-32c4f305-ba79-47db-b48f-91966b1cdcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:32.703802 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.704617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.705434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.708324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.709245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.711320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.728753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 848/1698] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:02:32.733710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.746865 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae91de9-9eb4-4caf-84a8-db789aa5b8bb admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.752591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.754720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.755221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.755752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] PATH_INFO: `/v3/projects/5067fcd621254353b5d0731c1655d680/users/06a9ea4fb5444cfca465d176db41f27e/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.767227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 851/1699] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:32.775237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4f305-ba79-47db-b48f-91966b1cdcc6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.800563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4f305-ba79-47db-b48f-91966b1cdcc6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.802568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5067fcd621254353b5d0731c1655d680, user_id=06a9ea4fb5444cfca465d176db41f27e, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.809539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.814781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.815625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.816014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.817008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.817642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:32.823477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:32.840303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 849/1700] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:32 2022] PUT /identity/v3/projects/5067fcd621254353b5d0731c1655d680/users/06a9ea4fb5444cfca465d176db41f27e/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:32.842969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.848019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:32.902785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 852/1701] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:32.908980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.920719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.042909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d302070-1f49-4199-bd4d-588555203e1a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.079022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334dff-2d49-45c9-80d3-df5e9c9e9071 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.093196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.094089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.094089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.094351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.096830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.103216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.130684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 853/1702] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:33.133137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.138158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:33.175547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dbdda-e18c-4ca9-9d71-457fef448ae0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.191720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.193607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.193607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.193607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] PATH_INFO: `/v3/users/d2b35881d3824c85badb4c7a67f3262c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.198794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b35881d3824c85badb4c7a67f3262c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.201242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.203237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.205499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.205799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.206085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] PATH_INFO: `/v3/projects/1b19d306684d4e9e80ba5db56585ca1c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.208196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b19d306684d4e9e80ba5db56585ca1c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.210722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.217098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.298614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b35881d3824c85badb4c7a67f3262c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.309341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b35881d3824c85badb4c7a67f3262c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.323131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b19d306684d4e9e80ba5db56585ca1c', 'request_id': 'req-22b2a201-6e66-4b66-9357-23ff5b26c809'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.339265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b35881d3824c85badb4c7a67f3262c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.340929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b35881d3824c85badb4c7a67f3262c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.342801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b35881d3824c85badb4c7a67f3262c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.346004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b19d306684d4e9e80ba5db56585ca1c', 'request_id': 'req-22b2a201-6e66-4b66-9357-23ff5b26c809'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.375185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b19d306684d4e9e80ba5db56585ca1c', 'request_id': 'req-22b2a201-6e66-4b66-9357-23ff5b26c809'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.387849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 854/1703] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:33 2022] DELETE /identity/v3/users/d2b35881d3824c85badb4c7a67f3262c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:33.391310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.396009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:33.409366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] The token cache is being invalidate because project 1b19d306684d4e9e80ba5db56585ca1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:33.409686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.413664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:33.420522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 850/1704] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:32 2022] DELETE /identity/v3/projects/1b19d306684d4e9e80ba5db56585ca1c => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:33.420522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.422617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:33.585242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c190-77b4-4acc-acd4-bdf7d60a8ec9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.598320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e42186-7ce5-4308-8e59-41648a5ef49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.598878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e42186-7ce5-4308-8e59-41648a5ef49f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.599160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e42186-7ce5-4308-8e59-41648a5ef49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.599466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e42186-7ce5-4308-8e59-41648a5ef49f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.603839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e42186-7ce5-4308-8e59-41648a5ef49f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.605906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e42186-7ce5-4308-8e59-41648a5ef49f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.612879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.618266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2a201-6e66-4b66-9357-23ff5b26c809 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.657109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a3e73-b94f-469b-be53-e9514b807285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.658223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a3e73-b94f-469b-be53-e9514b807285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.658723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a3e73-b94f-469b-be53-e9514b807285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.659184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a3e73-b94f-469b-be53-e9514b807285 admin admin] PATH_INFO: `/v3/projects/5067fcd621254353b5d0731c1655d680/users/06a9ea4fb5444cfca465d176db41f27e/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.664181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 855/1705] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:33 2022] POST /identity/v3/users => generated 296 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:33.664968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a3e73-b94f-469b-be53-e9514b807285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5067fcd621254353b5d0731c1655d680, user_id=06a9ea4fb5444cfca465d176db41f27e, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.665667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.666007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.666286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.668458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a3e73-b94f-469b-be53-e9514b807285 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.685366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 851/1706] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:33 2022] PUT /identity/v3/projects/5067fcd621254353b5d0731c1655d680/users/06a9ea4fb5444cfca465d176db41f27e/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:33.686708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6e7c5-4c04-4cb0-aff8-8419047853a1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.687058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6e7c5-4c04-4cb0-aff8-8419047853a1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.687334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6e7c5-4c04-4cb0-aff8-8419047853a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.706491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:33.710216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:33.731659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2ee6e7c5-4c04-4cb0-aff8-8419047853a1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:33.731659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2ee6e7c5-4c04-4cb0-aff8-8419047853a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:33.821936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 856/1707] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:33.827109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.830065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:33.834796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 852/1708] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:33.836142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.836432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.836707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.847548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2df6-bbc8-4178-8df0-9c0cc1238fc0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.860341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de87d0-aa0d-402f-8ef7-5f5fc401a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.862323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de87d0-aa0d-402f-8ef7-5f5fc401a218 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.862826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de87d0-aa0d-402f-8ef7-5f5fc401a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.864188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de87d0-aa0d-402f-8ef7-5f5fc401a218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.865171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de87d0-aa0d-402f-8ef7-5f5fc401a218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.867869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de87d0-aa0d-402f-8ef7-5f5fc401a218 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.876609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] MFA Rules not processed for user `06a9ea4fb5444cfca465d176db41f27e`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:33.877855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:33.899206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 857/1709] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:33.901007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28006db-12e8-48a0-a723-0e5fe165605d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.901300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28006db-12e8-48a0-a723-0e5fe165605d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.901577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28006db-12e8-48a0-a723-0e5fe165605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.921117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.977422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d28006db-12e8-48a0-a723-0e5fe165605d None None] MFA Rules not processed for user `56a554beedec4915b6ad1066823c8d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:33.981012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d28006db-12e8-48a0-a723-0e5fe165605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:34.044237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 853/1710] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:34.045240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.046866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:34.228997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c17a4-55e9-44b2-a03d-363a392a53bb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.246733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 858/1711] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:34.248924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.249679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.250583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.255301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.256227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.256770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.257227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.280955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.282904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.300870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:34.303424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:34.311732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 854/1712] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:34.312774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.324787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b1c22-8083-4737-8a3a-6778e78e2dd5 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.330321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 859/1713] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:34.331412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.332985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:34.354673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8947-01e6-49b6-ae82-0e1dd5dd1f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.355653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8947-01e6-49b6-ae82-0e1dd5dd1f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.355943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8947-01e6-49b6-ae82-0e1dd5dd1f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.356224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8947-01e6-49b6-ae82-0e1dd5dd1f1c admin admin] PATH_INFO: `/v3/projects/a59c35e4223340348ba7e5a1c1d057f2/users/bf09251d78654c38a6c90f2a1f74a8c3/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.444242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8947-01e6-49b6-ae82-0e1dd5dd1f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59c35e4223340348ba7e5a1c1d057f2, user_id=bf09251d78654c38a6c90f2a1f74a8c3, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.449831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8947-01e6-49b6-ae82-0e1dd5dd1f1c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.461810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.483762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 855/1714] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:34 2022] PUT /identity/v3/projects/a59c35e4223340348ba7e5a1c1d057f2/users/bf09251d78654c38a6c90f2a1f74a8c3/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:34.485282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a9e4e-8598-4024-a25a-fa298ec63b1f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.485735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a9e4e-8598-4024-a25a-fa298ec63b1f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.486178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a9e4e-8598-4024-a25a-fa298ec63b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.572530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536dd3a-a7f1-43d5-90c9-3ba396a51c1f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.574521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e58a9e4e-8598-4024-a25a-fa298ec63b1f None None] MFA Rules not processed for user `56a554beedec4915b6ad1066823c8d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:34.575986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e58a9e4e-8598-4024-a25a-fa298ec63b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:34.736517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.737986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.738284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.738566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.765476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 856/1715] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:34.766847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a9e4e-8598-4024-a25a-fa298ec63b1f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.884556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.911747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.917502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.950720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 860/1716] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:34.954137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.956334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a9e4e-8598-4024-a25a-fa298ec63b1f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.976212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.977276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.979022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.980377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.981802 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:34.984447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.984641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e12923-47e2-4bd5-9e12-9372daf86d24 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.990037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.001635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.004489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.004853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.005203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.005879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.009453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.035911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 857/1717] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.039176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.044823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 861/1718] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:35.045579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:35.052663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.077966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158ec4-87fa-4392-8812-777488a8db6e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.099459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.100754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.101040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.101310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.101954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.113531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.198831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 862/1719] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:35 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:35.202682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.244927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da059-511c-4c58-88d3-96468f024b51 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.308088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.332708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc80c7-aaaa-4e77-babf-a53f8ebeebd2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.343027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.344058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.345650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.346025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] PATH_INFO: `/v3/projects/a59c35e4223340348ba7e5a1c1d057f2/users/bf09251d78654c38a6c90f2a1f74a8c3/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.349535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59c35e4223340348ba7e5a1c1d057f2, user_id=bf09251d78654c38a6c90f2a1f74a8c3, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.356525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.387558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 863/1720] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:35 2022] PUT /identity/v3/projects/a59c35e4223340348ba7e5a1c1d057f2/users/bf09251d78654c38a6c90f2a1f74a8c3/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:35.387558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.408651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.409366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.409897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.410097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.500358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecf594-2dc7-46ef-9e4c-3e0025f429c2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.501387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.507793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.508804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.509208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.509657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.514524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:35.524697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.549269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 858/1721] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:35.550165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.554178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:35.596487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf673f-90b8-4821-8454-63ee0d9d5306 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.705615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.724007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.755357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 864/1722] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:35.764439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.776951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.777618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.777895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.778175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] PATH_INFO: `/v3/projects/20fcdfc6f40b46adbf6d06c914bd2ffc` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.784246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fcdfc6f40b46adbf6d06c914bd2ffc)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.790826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.806241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebbf25-42e1-4dae-a848-6c28ccc103d0 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.821419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f8e68-94c2-4807-9b90-44e2b78854f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.822490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8e68-94c2-4807-9b90-44e2b78854f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.823125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8e68-94c2-4807-9b90-44e2b78854f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.823816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8e68-94c2-4807-9b90-44e2b78854f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.825170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f8e68-94c2-4807-9b90-44e2b78854f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.829223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f8e68-94c2-4807-9b90-44e2b78854f9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.857841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.904923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 865/1723] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:35 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:35.909016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e4ab-9bbe-4aef-aef4-eef1bb6f5271 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.910327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e4ab-9bbe-4aef-aef4-eef1bb6f5271 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.912431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e4ab-9bbe-4aef-aef4-eef1bb6f5271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.959445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fcdfc6f40b46adbf6d06c914bd2ffc', 'request_id': 'req-dbcc6ced-4cea-4664-b71f-e25f5f521a40'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:35.977755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fcdfc6f40b46adbf6d06c914bd2ffc', 'request_id': 'req-dbcc6ced-4cea-4664-b71f-e25f5f521a40'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:36.026726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fcdfc6f40b46adbf6d06c914bd2ffc', 'request_id': 'req-dbcc6ced-4cea-4664-b71f-e25f5f521a40'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:36.034428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0479e4ab-9bbe-4aef-aef4-eef1bb6f5271 None None] MFA Rules not processed for user `bf09251d78654c38a6c90f2a1f74a8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:36.040051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0479e4ab-9bbe-4aef-aef4-eef1bb6f5271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.049915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.060602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] The token cache is being invalidate because project 20fcdfc6f40b46adbf6d06c914bd2ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:36.060906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:36.065169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-dbcc6ced-4cea-4664-b71f-e25f5f521a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:36.081775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 859/1724] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:35 2022] DELETE /identity/v3/projects/20fcdfc6f40b46adbf6d06c914bd2ffc => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:36.081775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.081775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.081775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.173489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] MFA Rules not processed for user `eaef8f72768e4d1daf21257fdaf580d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:36.175628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.285477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 866/1725] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:36.288203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4cbb-d621-4f21-955f-7e2d92549e07 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.288203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4cbb-d621-4f21-955f-7e2d92549e07 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.288203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4cbb-d621-4f21-955f-7e2d92549e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.361565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 860/1726] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:36.361565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.361565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.377482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e9ae4cbb-d621-4f21-955f-7e2d92549e07 None None] MFA Rules not processed for user `e24e6fb4c69d48f7b2972b53fb8ea4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:36.380162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9ae4cbb-d621-4f21-955f-7e2d92549e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.539067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb8e4a-bca2-4c20-85c6-1430c1d67de6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.559299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 867/1727] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:36.559299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.559299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.559299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.559902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.566051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.566516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.572262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.573026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.574711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.600939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 861/1728] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:36.601833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.606695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:36.613144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.622054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.631344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 868/1729] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:36.632303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.634991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.748528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeb9dd-b11c-46bf-a39b-3c3d32d5d097 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.756076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c60f85-ca49-482c-87a7-26eb91173ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.756753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c60f85-ca49-482c-87a7-26eb91173ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.757037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c60f85-ca49-482c-87a7-26eb91173ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.757311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c60f85-ca49-482c-87a7-26eb91173ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.766587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.777566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c60f85-ca49-482c-87a7-26eb91173ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.779034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c60f85-ca49-482c-87a7-26eb91173ad9 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.804846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 862/1730] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:36.806041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a92c34-aa29-4ff2-a9b7-ef4f5dc49014 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.806334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a92c34-aa29-4ff2-a9b7-ef4f5dc49014 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.810004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a92c34-aa29-4ff2-a9b7-ef4f5dc49014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.832322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009763d4-ecfb-45e8-80d8-ea34cfb9c30a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.845736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-70a92c34-aa29-4ff2-a9b7-ef4f5dc49014 None None] MFA Rules not processed for user `e24e6fb4c69d48f7b2972b53fb8ea4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:36.847093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-70a92c34-aa29-4ff2-a9b7-ef4f5dc49014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:36.855954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 863/1731] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:36.856819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a92c34-aa29-4ff2-a9b7-ef4f5dc49014 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.856979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820aeef3-eb3e-4712-b400-b3f32696cd11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.857616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aeef3-eb3e-4712-b400-b3f32696cd11 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.861454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aeef3-eb3e-4712-b400-b3f32696cd11 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.861655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aeef3-eb3e-4712-b400-b3f32696cd11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.869444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a92c34-aa29-4ff2-a9b7-ef4f5dc49014 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.876037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0ba75-dd1c-4dd5-9009-ab334f1db433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.876824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0ba75-dd1c-4dd5-9009-ab334f1db433 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.877111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0ba75-dd1c-4dd5-9009-ab334f1db433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.877384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0ba75-dd1c-4dd5-9009-ab334f1db433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.879545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0ba75-dd1c-4dd5-9009-ab334f1db433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.886829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0ba75-dd1c-4dd5-9009-ab334f1db433 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.907097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820aeef3-eb3e-4712-b400-b3f32696cd11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.909010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820aeef3-eb3e-4712-b400-b3f32696cd11 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.911591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 864/1732] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:36.913286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.913794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.914279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.944583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 869/1733] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:36.946320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2bff-b28c-4868-8f68-543ccc2ffd30 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.946611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2bff-b28c-4868-8f68-543ccc2ffd30 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.946887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d2bff-b28c-4868-8f68-543ccc2ffd30 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.952191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 870/1734] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 23:02:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:36.958489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e852-e2fa-4c32-9a5c-7f67ccbfa217 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.958755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e852-e2fa-4c32-9a5c-7f67ccbfa217 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.959020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e852-e2fa-4c32-9a5c-7f67ccbfa217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.961468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:36.966138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.001981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 865/1735] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:37.017569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.029252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.039789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7162e852-e2fa-4c32-9a5c-7f67ccbfa217 None None] MFA Rules not processed for user `85cd8a90230b4c69886534be67acd943`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:37.041227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7162e852-e2fa-4c32-9a5c-7f67ccbfa217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.048618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:37.057237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a78a7-58ed-4ac8-8fe6-26ceb4492796 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.070797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac023fa9-9a1d-473d-bcf8-5ba59e920bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.071437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac023fa9-9a1d-473d-bcf8-5ba59e920bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.071806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac023fa9-9a1d-473d-bcf8-5ba59e920bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.072081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac023fa9-9a1d-473d-bcf8-5ba59e920bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.080227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ac023fa9-9a1d-473d-bcf8-5ba59e920bb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.240284 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac023fa9-9a1d-473d-bcf8-5ba59e920bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.242347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac023fa9-9a1d-473d-bcf8-5ba59e920bb9 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.263206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 871/1736] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:37.263206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162e852-e2fa-4c32-9a5c-7f67ccbfa217 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.284900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 866/1737] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:37.287792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d002414-57a4-4ef2-acf9-bb5dbf477bbe None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.287888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162e852-e2fa-4c32-9a5c-7f67ccbfa217 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.287979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d002414-57a4-4ef2-acf9-bb5dbf477bbe None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.287979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d002414-57a4-4ef2-acf9-bb5dbf477bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.293548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.294395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.294794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.295187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] PATH_INFO: `/v3/projects/f1916937b68042629ba0f61daf0038eb/users/666f218097b344438c70cc857d5ab5f5/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.339941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1916937b68042629ba0f61daf0038eb, user_id=666f218097b344438c70cc857d5ab5f5, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.344248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.380667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 872/1738] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:37 2022] PUT /identity/v3/projects/f1916937b68042629ba0f61daf0038eb/users/666f218097b344438c70cc857d5ab5f5/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:37.380667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.403646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1d002414-57a4-4ef2-acf9-bb5dbf477bbe None None] MFA Rules not processed for user `c0e10b1ef7454167bd1c606ef4f70bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:37.405653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1d002414-57a4-4ef2-acf9-bb5dbf477bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.528037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5034c-6db3-477a-9ebf-bab5db171567 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.534981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d90ce-74a7-4463-99d9-d62fc83600c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.535698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d90ce-74a7-4463-99d9-d62fc83600c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.538191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d90ce-74a7-4463-99d9-d62fc83600c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.538528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d90ce-74a7-4463-99d9-d62fc83600c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.538832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 867/1739] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:02:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:37.540954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d90ce-74a7-4463-99d9-d62fc83600c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:37.541147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d90ce-74a7-4463-99d9-d62fc83600c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.541327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d002414-57a4-4ef2-acf9-bb5dbf477bbe None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.544700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d90ce-74a7-4463-99d9-d62fc83600c5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.579779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 873/1740] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:37.579779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.579779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.579779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.648893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] MFA Rules not processed for user `666f218097b344438c70cc857d5ab5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:37.651575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.713528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d002414-57a4-4ef2-acf9-bb5dbf477bbe None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.726072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:37.729701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.733441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.734962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.736863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.805729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 874/1741] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:37.807921 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.810278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.821570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:37.836563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610f90-ff53-4b07-b8d8-8f072e4bd47d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.845918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.846954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.848766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.849342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.864632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.870997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.874852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.876996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.901304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 868/1742] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:37.905132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.922013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 875/1743] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:37.925560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.933357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351f7bf-2651-4491-9a6d-7c266ed57f3a service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.948600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.951317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.953346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.954124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.958016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.961660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379a5b0-5bca-46f6-bdb3-2b66cca986c8 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.964144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.977668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.978417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.978714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.979014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.979740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.989969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.081971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 869/1744] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:37 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:38.086055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.099388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 876/1745] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:37 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:38.100687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.255073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e954ae-16e3-42b2-bcfe-64b043a3a1fa admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.265332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.265975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.266262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.266535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.267992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.269599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.304342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdb2d3-c748-43c2-bf64-bcf131e80b3d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.308522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 870/1746] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:38 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:02:38.318694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.326240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.328331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.330834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e65fc-65a4-479d-9a03-0a431c997949 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.337854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.340845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.341716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.344548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.346300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.346996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.347286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.347592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.349196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.359395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.370088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 871/1747] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:38.372751 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.384704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658d6f0-3455-41d8-a3ac-e45d000e1ea1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.393083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.394456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.396405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.397673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.398582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.402291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.420357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 877/1748] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:38 2022] POST /identity/v3/users => generated 291 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:38.421618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.425939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 872/1749] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:38.427731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.447548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f8ed-da68-44cf-b366-f36d76314103 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.448215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf8f1-3418-4852-9f77-2eb4be400965 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.458325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.458964 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.459255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.459597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.460259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.461439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8364b16-0deb-4a48-b551-84064f4ba09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.462210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8364b16-0deb-4a48-b551-84064f4ba09f admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.462502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8364b16-0deb-4a48-b551-84064f4ba09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.462779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8364b16-0deb-4a48-b551-84064f4ba09f admin admin] PATH_INFO: `/v3/projects/1e3d66e24fa5401ea6853092fc4e4dff/users/60ccdb8c2dc94916920ffa4efeb7c327/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.466292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.497880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 878/1750] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:38.498074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8364b16-0deb-4a48-b551-84064f4ba09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3d66e24fa5401ea6853092fc4e4dff, user_id=60ccdb8c2dc94916920ffa4efeb7c327, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.499009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.500125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8364b16-0deb-4a48-b551-84064f4ba09f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.520006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e17f7a-1336-48a0-a4ce-ee081fc0d069 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.524555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 873/1751] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:38 2022] PUT /identity/v3/projects/1e3d66e24fa5401ea6853092fc4e4dff/users/60ccdb8c2dc94916920ffa4efeb7c327/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:38.542421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a27cf-b214-4fd2-b10f-da9ba062e6a7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.542719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a27cf-b214-4fd2-b10f-da9ba062e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.543000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a27cf-b214-4fd2-b10f-da9ba062e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.608444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f21a27cf-b214-4fd2-b10f-da9ba062e6a7 None None] MFA Rules not processed for user `60ccdb8c2dc94916920ffa4efeb7c327`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:38.610025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f21a27cf-b214-4fd2-b10f-da9ba062e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:38.673635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdf8dc-64b7-470b-9c61-1b408387bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.674654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdf8dc-64b7-470b-9c61-1b408387bae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.675080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdf8dc-64b7-470b-9c61-1b408387bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.675767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdf8dc-64b7-470b-9c61-1b408387bae6 admin admin] PATH_INFO: `/v3/projects/de71a78d627c44049cdb3ca00a12be5b/users/ba6ce74915b34c2e9a1d129083f86795/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.690593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:38.731723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdf8dc-64b7-470b-9c61-1b408387bae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de71a78d627c44049cdb3ca00a12be5b, user_id=ba6ce74915b34c2e9a1d129083f86795, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.732851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdf8dc-64b7-470b-9c61-1b408387bae6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.755016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:38.760167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 874/1752] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:38.771824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 879/1753] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:38 2022] PUT /identity/v3/projects/de71a78d627c44049cdb3ca00a12be5b/users/ba6ce74915b34c2e9a1d129083f86795/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:38.776206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a27cf-b214-4fd2-b10f-da9ba062e6a7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.783048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985216-4d61-4285-9e68-e1438aaeb527 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.783532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985216-4d61-4285-9e68-e1438aaeb527 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.785186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985216-4d61-4285-9e68-e1438aaeb527 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.786954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 880/1754] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 23:02:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:38.795762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.797723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.799761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.815803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a27cf-b214-4fd2-b10f-da9ba062e6a7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.844223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] MFA Rules not processed for user `c0e10b1ef7454167bd1c606ef4f70bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:38.846282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:38.914561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.915329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.915655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.915935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.916644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.918329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.935545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 875/1755] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:02:38.938641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.962132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052d378-7f1d-4d89-a3c3-5ebc1a598942 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.974245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8944a-5c1d-4e7d-8047-25a291605dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.974999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8944a-5c1d-4e7d-8047-25a291605dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.978674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8944a-5c1d-4e7d-8047-25a291605dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.979025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8944a-5c1d-4e7d-8047-25a291605dfe admin admin] PATH_INFO: `/v3/projects/de71a78d627c44049cdb3ca00a12be5b/users/ba6ce74915b34c2e9a1d129083f86795/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.982733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8944a-5c1d-4e7d-8047-25a291605dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=de71a78d627c44049cdb3ca00a12be5b, user_id=ba6ce74915b34c2e9a1d129083f86795, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.985915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8944a-5c1d-4e7d-8047-25a291605dfe admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:39.000893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 876/1756] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:38 2022] PUT /identity/v3/projects/de71a78d627c44049cdb3ca00a12be5b/users/ba6ce74915b34c2e9a1d129083f86795/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:39.015951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404abbad-a3dc-44a2-9fa5-42c68ce6cbf2 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.015951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404abbad-a3dc-44a2-9fa5-42c68ce6cbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.016637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404abbad-a3dc-44a2-9fa5-42c68ce6cbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.026587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 881/1757] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:02:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:39.049454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.052610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:39.083423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd21ee5-1715-4e0e-ad08-d16bd049539a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.139531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-404abbad-a3dc-44a2-9fa5-42c68ce6cbf2 None None] MFA Rules not processed for user `ba6ce74915b34c2e9a1d129083f86795`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:39.148428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-404abbad-a3dc-44a2-9fa5-42c68ce6cbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:39.331300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.334212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.338154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.339412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.402981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:39.404748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:39.435037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 882/1758] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:02:39.438817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.457594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:39.507205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 877/1759] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:39.618343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404abbad-a3dc-44a2-9fa5-42c68ce6cbf2 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.776482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eb5be-feff-4740-8666-9494c6595ef0 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.791765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.792589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.793011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.793424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.808915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404abbad-a3dc-44a2-9fa5-42c68ce6cbf2 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.819850 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.820485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.820766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.823368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.842791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:39.844879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:39.861257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.869495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 878/1760] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:39.870381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.875473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:39.906377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.952411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:39.959284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.000644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 883/1761] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:40.024840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6859d1-ad86-4f2d-8ef3-89173e2a0489 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.033154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.034281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.034604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.035933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.045802 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:40.080020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.081934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.114887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 879/1762] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:40.191040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.211754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7beb553-30c7-42b1-bccb-6c7ee0f1ca8e service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.229794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085127-c4e0-41f6-ab1b-b70e6874bddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.232331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085127-c4e0-41f6-ab1b-b70e6874bddd service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.232934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085127-c4e0-41f6-ab1b-b70e6874bddd service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.233990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085127-c4e0-41f6-ab1b-b70e6874bddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.414813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff085127-c4e0-41f6-ab1b-b70e6874bddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.421847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff085127-c4e0-41f6-ab1b-b70e6874bddd service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.477393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 884/1763] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:41.638483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:41.815733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.816563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.817001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.887659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] MFA Rules not processed for user `e24e6fb4c69d48f7b2972b53fb8ea4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:41.893916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:41.894109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1745-19a0-446b-8dfb-5217109d9495 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:41.909462 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:41.910363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.910893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.911178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] PATH_INFO: `/v3/users/60ccdb8c2dc94916920ffa4efeb7c327` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.914363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ccdb8c2dc94916920ffa4efeb7c327)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:41.919590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:42.104071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ccdb8c2dc94916920ffa4efeb7c327'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.106798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ccdb8c2dc94916920ffa4efeb7c327'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.130451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ccdb8c2dc94916920ffa4efeb7c327'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.163923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ccdb8c2dc94916920ffa4efeb7c327'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.165411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3974f869-ac70-4c22-a6d3-d0b98af27165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ccdb8c2dc94916920ffa4efeb7c327'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.192055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 885/1764] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:42.193390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:42.204489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 880/1765] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:41 2022] DELETE /identity/v3/users/60ccdb8c2dc94916920ffa4efeb7c327 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:42.206024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:42.207359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:42.208002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:42.208505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:42.243950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:42.262035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:42.265261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0edd2-8ff7-474e-9eeb-acf0b0019f16 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:42.266131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:42.499050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:42.500497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:42.501719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:42.502886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] PATH_INFO: `/v3/users/71a11a30b6e84952ace83f2b7785a643` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:42.509602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a11a30b6e84952ace83f2b7785a643)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:42.513280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:42.566062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 881/1766] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:42.617573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:42.621568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:42.672701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:42.708220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a11a30b6e84952ace83f2b7785a643'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.733483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a11a30b6e84952ace83f2b7785a643'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.774504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a11a30b6e84952ace83f2b7785a643'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.781476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a11a30b6e84952ace83f2b7785a643'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.784180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f638f3e-1081-4702-be62-e12a9896e2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a11a30b6e84952ace83f2b7785a643'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:42.832160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 886/1767] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:42 2022] DELETE /identity/v3/users/71a11a30b6e84952ace83f2b7785a643 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:42.838692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:42.839144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:42.840509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf85bad-8b40-47af-a48c-bfc9b60a5e18 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:42.841400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:42.921786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:42.927212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:43.010838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:43.011761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.012048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.012877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.016200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:43.109305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 887/1768] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:43.160996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:43.163168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:43.185695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3096610-c6b6-4d2b-871a-66698f1657ab None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:43.186761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:43.189854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:43.206023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 882/1769] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:43.212429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dbdf8-e251-4d7d-ae1e-a229d7be1d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:43.213098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dbdf8-e251-4d7d-ae1e-a229d7be1d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.213882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dbdf8-e251-4d7d-ae1e-a229d7be1d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.213882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dbdf8-e251-4d7d-ae1e-a229d7be1d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.247288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dbdf8-e251-4d7d-ae1e-a229d7be1d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:43.248966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dbdf8-e251-4d7d-ae1e-a229d7be1d75 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:43.295515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 888/1770] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:44.234077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.236290 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:44.261910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287826-c71f-490c-b11b-7ff38ed3edc1 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.275201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.275975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.276270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.276572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] PATH_INFO: `/v3/users/666f218097b344438c70cc857d5ab5f5` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.278108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=666f218097b344438c70cc857d5ab5f5)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:44.282921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:44.297561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:44.394906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666f218097b344438c70cc857d5ab5f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:44.396076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666f218097b344438c70cc857d5ab5f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:44.416689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666f218097b344438c70cc857d5ab5f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:44.449245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666f218097b344438c70cc857d5ab5f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:44.449874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666f218097b344438c70cc857d5ab5f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:44.483205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 883/1771] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:44 2022] DELETE /identity/v3/users/666f218097b344438c70cc857d5ab5f5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:44.484718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c74b-a682-4cf8-bf4f-32b59ed8e9b3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.485232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c74b-a682-4cf8-bf4f-32b59ed8e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.485521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c74b-a682-4cf8-bf4f-32b59ed8e9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.542583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ffa7c74b-a682-4cf8-bf4f-32b59ed8e9b3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:44.548195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ffa7c74b-a682-4cf8-bf4f-32b59ed8e9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:44.778531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 889/1772] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:44.796804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.798296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:44.938029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e7b9a-b2f6-4e52-81ba-d10232938684 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.949274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.949906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.950186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.950453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.972737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:44.974185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:44.998949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871a18d-d17f-4c09-90f7-53aadbea6e4f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.999299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871a18d-d17f-4c09-90f7-53aadbea6e4f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.999573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871a18d-d17f-4c09-90f7-53aadbea6e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.030882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 884/1773] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:45.041328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3871a18d-d17f-4c09-90f7-53aadbea6e4f None None] MFA Rules not processed for user `85cd8a90230b4c69886534be67acd943`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:45.042975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3871a18d-d17f-4c09-90f7-53aadbea6e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:45.113131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.114623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:45.137944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8514a-8679-4edc-826e-3a18ed0b3f18 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.148489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.160233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.160892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.161157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.161431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] PATH_INFO: `/v3/projects/09300f5207f046658540d53cdde2b9dd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.171201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09300f5207f046658540d53cdde2b9dd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.181524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.239675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 890/1774] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:45.250479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb71bc-42f3-4687-a8c5-29f11e17a675 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.250772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb71bc-42f3-4687-a8c5-29f11e17a675 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.251047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb71bc-42f3-4687-a8c5-29f11e17a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.382083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09300f5207f046658540d53cdde2b9dd', 'request_id': 'req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.406608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09300f5207f046658540d53cdde2b9dd', 'request_id': 'req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.444623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09300f5207f046658540d53cdde2b9dd', 'request_id': 'req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.461343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] The token cache is being invalidate because project 09300f5207f046658540d53cdde2b9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:45.461701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.464131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.472389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 885/1775] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:45 2022] DELETE /identity/v3/projects/09300f5207f046658540d53cdde2b9dd => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:45.473478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.478261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:45.503104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c7eb71bc-42f3-4687-a8c5-29f11e17a675 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:45.505393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.509976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c7eb71bc-42f3-4687-a8c5-29f11e17a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:45.646109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7861e-7e17-40ea-99b8-1a88d04c4ca6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.663072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.665768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.666157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.671098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] PATH_INFO: `/v3/users/d1603025e10b44288d932fb1c0969e72` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.675801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1603025e10b44288d932fb1c0969e72)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.677828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.786248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 891/1776] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:02:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 536 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:45.789737 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.790288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.790796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.857810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] MFA Rules not processed for user `eaef8f72768e4d1daf21257fdaf580d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:45.859757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:45.877448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1603025e10b44288d932fb1c0969e72'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.879536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1603025e10b44288d932fb1c0969e72'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.912475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1603025e10b44288d932fb1c0969e72'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.946201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1603025e10b44288d932fb1c0969e72'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.947201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1603025e10b44288d932fb1c0969e72'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:45.986138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 886/1777] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:45 2022] DELETE /identity/v3/users/d1603025e10b44288d932fb1c0969e72 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:46.006905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.010209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:46.072578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 892/1778] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:46.401351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec4f8-38fe-4fd5-afa2-ab7da54d2148 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.419843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131e846-9061-419a-b747-b4c42be30853 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.419843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e846-9061-419a-b747-b4c42be30853 service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.424266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e846-9061-419a-b747-b4c42be30853 service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.424557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e846-9061-419a-b747-b4c42be30853 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.435433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d131e846-9061-419a-b747-b4c42be30853 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:46.471628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:46.580682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.582912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:46.607946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de810b-00b4-48fd-93b1-b7e9bcc53b5c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.657419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131e846-9061-419a-b747-b4c42be30853 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.661157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131e846-9061-419a-b747-b4c42be30853 service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.688928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 887/1779] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:02:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:02:46.851180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.851983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.852350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.852722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] PATH_INFO: `/v3/projects/1e3d66e24fa5401ea6853092fc4e4dff` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.861259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3d66e24fa5401ea6853092fc4e4dff)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.904311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.907746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:47.066994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3d66e24fa5401ea6853092fc4e4dff', 'request_id': 'req-74cb44ac-3e9a-40b1-bfb7-575e37828f23'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:47.076283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3d66e24fa5401ea6853092fc4e4dff', 'request_id': 'req-74cb44ac-3e9a-40b1-bfb7-575e37828f23'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:47.085320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3d66e24fa5401ea6853092fc4e4dff', 'request_id': 'req-74cb44ac-3e9a-40b1-bfb7-575e37828f23'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:47.108590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] The token cache is being invalidate because project 1e3d66e24fa5401ea6853092fc4e4dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:47.108942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:47.109566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-74cb44ac-3e9a-40b1-bfb7-575e37828f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:47.111341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 893/1780] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:46 2022] DELETE /identity/v3/projects/1e3d66e24fa5401ea6853092fc4e4dff => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:47.116735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adb02b-1590-461c-b044-13f2f3dd0ad0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:47.117447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adb02b-1590-461c-b044-13f2f3dd0ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:47.117942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adb02b-1590-461c-b044-13f2f3dd0ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:47.144923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-32adb02b-1590-461c-b044-13f2f3dd0ad0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:47.146544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-32adb02b-1590-461c-b044-13f2f3dd0ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:47.265879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 888/1781] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:47.274589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e47c-daa1-4112-93de-3048afde73de None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:47.275047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e47c-daa1-4112-93de-3048afde73de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:47.275534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e47c-daa1-4112-93de-3048afde73de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:47.328786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0e25e47c-daa1-4112-93de-3048afde73de None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:47.334998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e25e47c-daa1-4112-93de-3048afde73de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:47.363412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 894/1782] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:47.365124 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adb02b-1590-461c-b044-13f2f3dd0ad0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:47.380387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adb02b-1590-461c-b044-13f2f3dd0ad0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:47.397385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe770b12-7637-4088-9894-a497bbe0da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:47.397759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770b12-7637-4088-9894-a497bbe0da1b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:47.401689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770b12-7637-4088-9894-a497bbe0da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:47.403217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770b12-7637-4088-9894-a497bbe0da1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:47.407789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe770b12-7637-4088-9894-a497bbe0da1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:47.409149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe770b12-7637-4088-9894-a497bbe0da1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:47.414687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe770b12-7637-4088-9894-a497bbe0da1b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:47.458105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 889/1783] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:47.463306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25e47c-daa1-4112-93de-3048afde73de None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:47.475772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:47.476071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:47.476330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:47.477084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25e47c-daa1-4112-93de-3048afde73de None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:47.519481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:47.524675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:47.524965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:47.525244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:47.532806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:47.537596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] MFA Rules not processed for user `c0e10b1ef7454167bd1c606ef4f70bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:47.543307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:47.549454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:47.686019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 895/1784] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:47 2022] POST /identity/v3/projects => generated 369 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:47.690428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:47.767524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 890/1785] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 23:02:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:47.944693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbffb0d-f3a8-4973-997b-8a4730971da8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:47.957406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:47.959905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:47.961843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:47.963600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:47.963772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:47.964843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:47.965421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:47.967781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:47.982491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5d673-e00b-4cba-a99b-c26dc2ef4a3d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:48.031852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 896/1786] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:47 2022] POST /identity/v3/users => generated 285 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:48.039517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:48.061159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bd01b-eae3-4287-84c0-85a80434e11f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:48.071726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:48.072631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:48.073059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:48.074252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:48.075315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:48.077967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:48.113578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 897/1787] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:48.117573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:48.142928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1fff3-d89e-4f8e-9754-eaec079ee4da admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:48.154318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:48.155037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:48.155398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:48.155875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:48.156625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:48.161725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:48.210015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:48.226803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6d5fb-3bb3-4145-bea4-a827f7e5a4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:48.227362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6d5fb-3bb3-4145-bea4-a827f7e5a4de service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:48.229714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6d5fb-3bb3-4145-bea4-a827f7e5a4de service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:48.229941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6d5fb-3bb3-4145-bea4-a827f7e5a4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:48.237797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 898/1788] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:48.238895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:48.254870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6d5fb-3bb3-4145-bea4-a827f7e5a4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:48.257997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6d5fb-3bb3-4145-bea4-a827f7e5a4de service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:48.276103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a93fd-1408-4de0-85c7-fca39b1773db admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:48.280600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 891/1789] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:48.306061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:48.307691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:48.307691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:48.307691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] PATH_INFO: `/v3/projects/d69a5089d91c4195bb096c984c55718c/users/02cfc69cedc449d5b15abaffcefc26ac/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:48.347812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69a5089d91c4195bb096c984c55718c, user_id=02cfc69cedc449d5b15abaffcefc26ac, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:48.350866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:48.380132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 899/1790] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:48 2022] PUT /identity/v3/projects/d69a5089d91c4195bb096c984c55718c/users/02cfc69cedc449d5b15abaffcefc26ac/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:48.395455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:48.397560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:48.398113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:48.461358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] MFA Rules not processed for user `02cfc69cedc449d5b15abaffcefc26ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:48.465720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:48.789047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 892/1791] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:48.793618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:48.814698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:48.926951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:48.928773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:49.062349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903aa9a-1760-49ae-9e0c-59b43d80b222 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:49.082495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:49.083245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:49.083688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:49.084026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:49.085368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:49.097685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:49.183594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd5f18-c918-412e-b346-bcd65f794186 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:49.206655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:49.207120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 900/1792] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:48 2022] POST /identity/v3/projects => generated 369 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:49.207872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:49.208052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:49.211175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:49.212510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] PATH_INFO: `/v3/projects/ec8293c86d394e188ff2c8c88b2d491a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:49.216489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8293c86d394e188ff2c8c88b2d491a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:49.224608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:49.378271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8293c86d394e188ff2c8c88b2d491a', 'request_id': 'req-8a946adc-39f8-4e21-8de5-efbfba5d4f45'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.393189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8293c86d394e188ff2c8c88b2d491a', 'request_id': 'req-8a946adc-39f8-4e21-8de5-efbfba5d4f45'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.411293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8293c86d394e188ff2c8c88b2d491a', 'request_id': 'req-8a946adc-39f8-4e21-8de5-efbfba5d4f45'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.436948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] The token cache is being invalidate because project ec8293c86d394e188ff2c8c88b2d491a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:49.437298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.438963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.447103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 893/1793] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:48 2022] DELETE /identity/v3/projects/ec8293c86d394e188ff2c8c88b2d491a => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:49.447103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:49.451064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:49.486355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:49.498549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6889e-ded4-4eed-8c38-a2350546157d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:49.621905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a946adc-39f8-4e21-8de5-efbfba5d4f45 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:49.634873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:49.637532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:49.637866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:49.638105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] PATH_INFO: `/v3/projects/f1916937b68042629ba0f61daf0038eb` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:49.642783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1916937b68042629ba0f61daf0038eb)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:49.648896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:49.704676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:49.705613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:49.706282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:49.706816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:49.707593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:49.715883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:49.767005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 901/1794] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:49 2022] POST /identity/v3/users => generated 291 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:49.770164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:49.779436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:49.828642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08217-48b8-49c5-abb7-5cd5be53b079 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:49.830848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1916937b68042629ba0f61daf0038eb', 'request_id': 'req-523f4931-206a-42c0-a04f-c299abf11845'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.862454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1916937b68042629ba0f61daf0038eb', 'request_id': 'req-523f4931-206a-42c0-a04f-c299abf11845'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.889862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:49.898389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1916937b68042629ba0f61daf0038eb', 'request_id': 'req-523f4931-206a-42c0-a04f-c299abf11845'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.932060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] The token cache is being invalidate because project f1916937b68042629ba0f61daf0038eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:49.932364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.933707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:49.939265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 894/1795] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:49 2022] DELETE /identity/v3/projects/f1916937b68042629ba0f61daf0038eb => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:49.941738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:49.945080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:49.975676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:49.976351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:49.976665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:49.976939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] PATH_INFO: `/v3/users/bf09251d78654c38a6c90f2a1f74a8c3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:49.979353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf09251d78654c38a6c90f2a1f74a8c3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:49.983895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:50.092490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf09251d78654c38a6c90f2a1f74a8c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.103834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4931-206a-42c0-a04f-c299abf11845 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:50.107961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf09251d78654c38a6c90f2a1f74a8c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.117400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:50.118141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.119369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.121093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.123009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:50.125192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:50.142697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 895/1796] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:50.142697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:50.145906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:50.149718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf09251d78654c38a6c90f2a1f74a8c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.152234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf09251d78654c38a6c90f2a1f74a8c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.156295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf09251d78654c38a6c90f2a1f74a8c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.170810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2daf1-daf0-4c8c-9501-9e3c1238bea9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:50.177330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:50.178917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.179321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.179759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] PATH_INFO: `/v3/users/ba6ce74915b34c2e9a1d129083f86795` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.181283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6ce74915b34c2e9a1d129083f86795)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:50.183767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:50.197485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 902/1797] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:49 2022] DELETE /identity/v3/users/bf09251d78654c38a6c90f2a1f74a8c3 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:50.201373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:50.213663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:50.290429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:50.318203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6ce74915b34c2e9a1d129083f86795'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.320326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6ce74915b34c2e9a1d129083f86795'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.338964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6ce74915b34c2e9a1d129083f86795'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.367710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6ce74915b34c2e9a1d129083f86795'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.374452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6ce74915b34c2e9a1d129083f86795'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:50.393509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 896/1798] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:50 2022] DELETE /identity/v3/users/ba6ce74915b34c2e9a1d129083f86795 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:50.504344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bd103-430d-42cd-9347-9fc9bca6dcf7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:50.819855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:50.864290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:50.870795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.871095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.873985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] PATH_INFO: `/v3/projects/651dd53750904bb996c87e4677ed31be/users/62ea47314d9840e79771bf03b45fba6a/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.962967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=651dd53750904bb996c87e4677ed31be, user_id=62ea47314d9840e79771bf03b45fba6a, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:50.968313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.012502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 903/1799] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:50 2022] PUT /identity/v3/projects/651dd53750904bb996c87e4677ed31be/users/62ea47314d9840e79771bf03b45fba6a/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:51.016605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.018937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:51.073882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c92479-f1b0-4ff7-8996-b55786a956ed admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.341689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46abc8-a52e-4809-8135-4fdbbea6b2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.342709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46abc8-a52e-4809-8135-4fdbbea6b2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.343201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46abc8-a52e-4809-8135-4fdbbea6b2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.343749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46abc8-a52e-4809-8135-4fdbbea6b2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.344632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46abc8-a52e-4809-8135-4fdbbea6b2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.346478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46abc8-a52e-4809-8135-4fdbbea6b2c2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.366403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 897/1800] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:51.370763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.384863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40835635-7bf3-4b2f-a492-b8ba309e9a8c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.393996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.394975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.395515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.396094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] PATH_INFO: `/v3/projects/651dd53750904bb996c87e4677ed31be/users/62ea47314d9840e79771bf03b45fba6a/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.400192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=651dd53750904bb996c87e4677ed31be, user_id=62ea47314d9840e79771bf03b45fba6a, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.403950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.427851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 904/1801] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:51 2022] PUT /identity/v3/projects/651dd53750904bb996c87e4677ed31be/users/62ea47314d9840e79771bf03b45fba6a/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:51.442434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a560c7-2f3e-4372-8d16-5772b6c96e57 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.443240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a560c7-2f3e-4372-8d16-5772b6c96e57 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.449950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a560c7-2f3e-4372-8d16-5772b6c96e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.562748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c5a560c7-2f3e-4372-8d16-5772b6c96e57 None None] MFA Rules not processed for user `62ea47314d9840e79771bf03b45fba6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:51.564018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c5a560c7-2f3e-4372-8d16-5772b6c96e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:51.820921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 898/1802] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:51.835543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.843310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:51.989296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d19e43-5146-49ee-a81e-d3f5d552cbb2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.993540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.994274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.994853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.996131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:52.029854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:52.031436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:52.064157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 905/1803] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:52.193084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a560c7-2f3e-4372-8d16-5772b6c96e57 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:52.208642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a560c7-2f3e-4372-8d16-5772b6c96e57 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:52.216729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:52.219217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:52.219597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:52.222482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:52.235040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:52.348391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:52.416149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:52.418058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:52.445879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 899/1804] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:53.884761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:53.886511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:53.939379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:53.940946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:54.212875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f143f-0579-452e-9d9c-094469a640e8 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:54.229502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:54.236720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.237115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.237391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] PATH_INFO: `/v3/projects/de71a78d627c44049cdb3ca00a12be5b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.239619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da34948-228f-4fe7-8a1f-845f0cb5d5a2 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:54.240088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de71a78d627c44049cdb3ca00a12be5b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:54.245278 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:54.246609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:54.247524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.248089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.248628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] PATH_INFO: `/v3/projects/a59c35e4223340348ba7e5a1c1d057f2` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.250072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59c35e4223340348ba7e5a1c1d057f2)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:54.276831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:54.437035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59c35e4223340348ba7e5a1c1d057f2', 'request_id': 'req-46579b33-8731-4f55-8c2f-b8337e394f4b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.445262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59c35e4223340348ba7e5a1c1d057f2', 'request_id': 'req-46579b33-8731-4f55-8c2f-b8337e394f4b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.446410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de71a78d627c44049cdb3ca00a12be5b', 'request_id': 'req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.461267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de71a78d627c44049cdb3ca00a12be5b', 'request_id': 'req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.465691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59c35e4223340348ba7e5a1c1d057f2', 'request_id': 'req-46579b33-8731-4f55-8c2f-b8337e394f4b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.488632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de71a78d627c44049cdb3ca00a12be5b', 'request_id': 'req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.501258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] The token cache is being invalidate because project a59c35e4223340348ba7e5a1c1d057f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:54.502024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.504081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-46579b33-8731-4f55-8c2f-b8337e394f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.507057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:54.510658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 900/1805] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:53 2022] DELETE /identity/v3/projects/a59c35e4223340348ba7e5a1c1d057f2 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:54.517373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dbd0f-0974-445b-8fa8-45e5457f260e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.518350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dbd0f-0974-445b-8fa8-45e5457f260e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.518946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dbd0f-0974-445b-8fa8-45e5457f260e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.542877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] The token cache is being invalidate because project de71a78d627c44049cdb3ca00a12be5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:02:54.543338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.544798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22aeb517-1d98-4f1c-be60-3fb4e324e8b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:54.554026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 906/1806] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:02:53 2022] DELETE /identity/v3/projects/de71a78d627c44049cdb3ca00a12be5b => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:54.554532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b8fb-1477-4e8a-9e04-2a00655123aa None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.556546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b8fb-1477-4e8a-9e04-2a00655123aa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.557135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b8fb-1477-4e8a-9e04-2a00655123aa None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.560261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 907/1807] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:02:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:54.563688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943692f8-9c85-4b6a-aeb2-59b05a04bfc9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.565144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943692f8-9c85-4b6a-aeb2-59b05a04bfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.565497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943692f8-9c85-4b6a-aeb2-59b05a04bfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.600720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b18dbd0f-0974-445b-8fa8-45e5457f260e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:54.611950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b18dbd0f-0974-445b-8fa8-45e5457f260e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:54.631661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-943692f8-9c85-4b6a-aeb2-59b05a04bfc9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:54.635166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-943692f8-9c85-4b6a-aeb2-59b05a04bfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:54.819393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:54.871194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 901/1808] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:54.871194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d013d-5445-464e-9b02-55e2f546fd43 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.876697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d013d-5445-464e-9b02-55e2f546fd43 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.876697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d013d-5445-464e-9b02-55e2f546fd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.924602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 908/1809] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:54.926980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8854a93-9f9e-4198-8a09-6f63cb0c83fe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.927317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8854a93-9f9e-4198-8a09-6f63cb0c83fe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.931459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8854a93-9f9e-4198-8a09-6f63cb0c83fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.970429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-150d013d-5445-464e-9b02-55e2f546fd43 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:54.971890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-150d013d-5445-464e-9b02-55e2f546fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:55.011823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a8854a93-9f9e-4198-8a09-6f63cb0c83fe None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:55.013144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a8854a93-9f9e-4198-8a09-6f63cb0c83fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:55.052330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 909/1810] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:55.056327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca7c77-973e-4055-a647-10f3ed2d617d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.057820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca7c77-973e-4055-a647-10f3ed2d617d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.058191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca7c77-973e-4055-a647-10f3ed2d617d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.102605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-60ca7c77-973e-4055-a647-10f3ed2d617d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:55.108051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60ca7c77-973e-4055-a647-10f3ed2d617d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:55.143993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 910/1811] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:55.145061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca7c77-973e-4055-a647-10f3ed2d617d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.166981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca7c77-973e-4055-a647-10f3ed2d617d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.193196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.194355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.194725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.194928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.195678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:55.196022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.197880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.212035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 902/1812] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:02:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:55.212760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d013d-5445-464e-9b02-55e2f546fd43 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.253539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d013d-5445-464e-9b02-55e2f546fd43 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.253692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 911/1813] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:55.253692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.266161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efa572-9cec-452c-a32b-320d29b4b0fb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.273130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.273905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.274286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.275770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.276553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:55.278071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.281041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.327603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.330608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.336443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.338527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.339573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.350330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 903/1814] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:55.352215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.354923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.418780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc57bef-3bd8-4211-9a77-f30ff2db4a59 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.437821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.438445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.438726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.439000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.439671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.441585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.489365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 912/1815] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:55 2022] POST /identity/v3/projects => generated 373 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:55.493942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.514864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 904/1816] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:55 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:55.515835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.517632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:55.676148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5c076-edaf-4e76-8e97-79a13b5be90e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.681835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e5937-5760-48a8-8d44-3e2f8cfb4510 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.692923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.694103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.694401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.694684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.720003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.722837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.762244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 905/1817] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:55.763339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.854303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.897351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a39e0-64c8-485e-b99f-12673663db53 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.911028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.914331 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.914785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.915214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.916052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.919154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.940855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.966782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.969738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.970069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.970347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.971112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.981599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.985506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 906/1818] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:55 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:55.989621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.028566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3683d-4441-46d4-bede-a63c30b2fdbf admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.040169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 913/1819] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:55 2022] POST /identity/v3/users => generated 287 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:56.041650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.049714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.050959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.051569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.052014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.055725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.058849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.065659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee03b-c187-496f-8786-ad9c62f420c9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.078050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.079268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.079581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.079949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.084496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.086636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 907/1820] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:02:56.088292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.090827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.115967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 914/1821] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:02:56.117171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.119214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad71e5-b9e3-4ff1-9227-e49a9788c874 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.133733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.134682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.135121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.135581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.136406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.138681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.140503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62542c44-f636-4a83-8503-5ec3bbebae43 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.150745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.151646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.151928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.152200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.152857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.154881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.172363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 915/1822] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:56.173840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.175358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 908/1823] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:56.179630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.193538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae050af8-c78a-40b9-9ad7-29457d19ef2e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.200635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee56ae-82b7-4717-8d63-4b426f32d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.201539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee56ae-82b7-4717-8d63-4b426f32d06d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.201963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee56ae-82b7-4717-8d63-4b426f32d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.202387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee56ae-82b7-4717-8d63-4b426f32d06d admin admin] PATH_INFO: `/v3/projects/f8456e899a194472bf935a740f16f11c/users/8c9e5cd314134970bbd41da428f00112/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.217180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06033ce4-3c18-43f6-889b-e8cfcf5d5028 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.239359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5d809-9867-465e-bc9a-f5df1eab4a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.240364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5d809-9867-465e-bc9a-f5df1eab4a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.240807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5d809-9867-465e-bc9a-f5df1eab4a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.241242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5d809-9867-465e-bc9a-f5df1eab4a0d admin admin] PATH_INFO: `/v3/projects/15cf90bead99458badb5625fababfebc/users/d5e7cd285e084f4cb1e5249489216992/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.242969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee56ae-82b7-4717-8d63-4b426f32d06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8456e899a194472bf935a740f16f11c, user_id=8c9e5cd314134970bbd41da428f00112, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.247645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee56ae-82b7-4717-8d63-4b426f32d06d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.268213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5d809-9867-465e-bc9a-f5df1eab4a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=15cf90bead99458badb5625fababfebc, user_id=d5e7cd285e084f4cb1e5249489216992, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.271211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5d809-9867-465e-bc9a-f5df1eab4a0d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.273104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 909/1824] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:56 2022] PUT /identity/v3/projects/f8456e899a194472bf935a740f16f11c/users/8c9e5cd314134970bbd41da428f00112/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:56.283253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.283580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.283821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.305329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 916/1825] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:56 2022] PUT /identity/v3/projects/15cf90bead99458badb5625fababfebc/users/d5e7cd285e084f4cb1e5249489216992/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:56.310599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96030467-ca0d-4a8c-9441-5ef5e8f2c6ab None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.311285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96030467-ca0d-4a8c-9441-5ef5e8f2c6ab None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.312152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96030467-ca0d-4a8c-9441-5ef5e8f2c6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.362159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-96030467-ca0d-4a8c-9441-5ef5e8f2c6ab None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:56.363646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-96030467-ca0d-4a8c-9441-5ef5e8f2c6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:56.452007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] MFA Rules not processed for user `8c9e5cd314134970bbd41da428f00112`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:56.453604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:56.583973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 917/1826] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:56.704934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 910/1827] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:02:56.706263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96030467-ca0d-4a8c-9441-5ef5e8f2c6ab None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.731302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96030467-ca0d-4a8c-9441-5ef5e8f2c6ab None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.771741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.772626 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.773852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.775005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.775455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:56.776791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.778062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.781393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.866808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 918/1828] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:02:56 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:56.871380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.944496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2574b6-5ee3-425b-abe3-dbcada1dad7b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.045189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.094702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dde22e-bc25-4d54-be78-46c383a8d9a6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.109729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.110937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.112870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.118407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.138795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.140447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.143117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.143584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.146520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.154036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.185656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.187566 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.215915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 919/1829] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:02:56 2022] POST /identity/v3/users => generated 300 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:57.217423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.241615 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 911/1830] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:57.245775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0008a19-53fe-4639-8c77-42651963d296 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.295397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d5883-9b75-4cfe-b839-705d32c75367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.296356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d5883-9b75-4cfe-b839-705d32c75367 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.299649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d5883-9b75-4cfe-b839-705d32c75367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.300395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d5883-9b75-4cfe-b839-705d32c75367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.302001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d5883-9b75-4cfe-b839-705d32c75367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.305239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d5883-9b75-4cfe-b839-705d32c75367 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.315762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.357066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 920/1831] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:57.357572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.387884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bb671-53df-4348-ba66-f12068360616 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.388786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e1c7e-fd3c-455f-87e4-731618694ca2 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.388915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bb671-53df-4348-ba66-f12068360616 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.389307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bb671-53df-4348-ba66-f12068360616 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.391477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 921/1832] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:02:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:57.402838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.403535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.403883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.404196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] PATH_INFO: `/v3/projects/92da77fbd4f244a4ac70be3a1b5bb2d5/users/13ba5b4fc5c0417c99a2b946dbe72845/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.441767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92da77fbd4f244a4ac70be3a1b5bb2d5, user_id=13ba5b4fc5c0417c99a2b946dbe72845, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.452069 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.487561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 912/1833] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:57 2022] PUT /identity/v3/projects/92da77fbd4f244a4ac70be3a1b5bb2d5/users/13ba5b4fc5c0417c99a2b946dbe72845/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:57.493944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bb671-53df-4348-ba66-f12068360616 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.751854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bb671-53df-4348-ba66-f12068360616 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.763475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.775752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.776116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.776430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.777465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.783188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.798577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 922/1834] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:02:57.800568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.803902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.806438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:57.836663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f9867-5380-4618-9711-67c1859cf9a9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.852342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ae9ae-45e2-42b7-b63b-82a393d20713 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.861060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da492f1a-403d-4dc4-a6ee-b434d41c9d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.861454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da492f1a-403d-4dc4-a6ee-b434d41c9d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.861920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da492f1a-403d-4dc4-a6ee-b434d41c9d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.862222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da492f1a-403d-4dc4-a6ee-b434d41c9d7a admin admin] PATH_INFO: `/v3/projects/92da77fbd4f244a4ac70be3a1b5bb2d5/users/13ba5b4fc5c0417c99a2b946dbe72845/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.872232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da492f1a-403d-4dc4-a6ee-b434d41c9d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=92da77fbd4f244a4ac70be3a1b5bb2d5, user_id=13ba5b4fc5c0417c99a2b946dbe72845, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.873609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.875280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da492f1a-403d-4dc4-a6ee-b434d41c9d7a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.879053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.879342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.879772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] PATH_INFO: `/v3/users/02cfc69cedc449d5b15abaffcefc26ac` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.882655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02cfc69cedc449d5b15abaffcefc26ac)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.886206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.911222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 923/1835] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:02:57 2022] PUT /identity/v3/projects/92da77fbd4f244a4ac70be3a1b5bb2d5/users/13ba5b4fc5c0417c99a2b946dbe72845/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:57.914526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa6099-4838-4125-af33-d3aa1bdaf6e8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.917890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa6099-4838-4125-af33-d3aa1bdaf6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.918362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa6099-4838-4125-af33-d3aa1bdaf6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.954624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-aefa6099-4838-4125-af33-d3aa1bdaf6e8 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:57.956702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aefa6099-4838-4125-af33-d3aa1bdaf6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:57.991058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02cfc69cedc449d5b15abaffcefc26ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:57.994256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02cfc69cedc449d5b15abaffcefc26ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.024518 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02cfc69cedc449d5b15abaffcefc26ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.072306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02cfc69cedc449d5b15abaffcefc26ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.075757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-79167ebc-e37d-414c-8c6e-29e5d3b6a4f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02cfc69cedc449d5b15abaffcefc26ac'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.114391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 913/1836] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:57 2022] DELETE /identity/v3/users/02cfc69cedc449d5b15abaffcefc26ac => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:58.116084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.119498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.120056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.206583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 924/1837] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:02:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:58.216141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.217647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.224714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.246776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] MFA Rules not processed for user `13ba5b4fc5c0417c99a2b946dbe72845`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:58.261199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:58.317052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:58.318368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:58.472613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 925/1838] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:58.473708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.476357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:58.484992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 914/1839] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:58.490644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.492901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:58.500497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3935ef-411b-4866-8783-d8def3f05893 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.506124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.506910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.507164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.507760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] PATH_INFO: `/v3/users/fe043673440148389343579de277473d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.509133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe043673440148389343579de277473d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.511611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.518023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.519146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.664472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe043673440148389343579de277473d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.686558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe043673440148389343579de277473d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.697632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057612a-c7e8-437d-90dc-b416a7e2bb76 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.723783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.725327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.725796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.726383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.734878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe043673440148389343579de277473d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.735863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe043673440148389343579de277473d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.736789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe043673440148389343579de277473d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:58.759108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.760990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.786762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 926/1840] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:58 2022] DELETE /identity/v3/users/fe043673440148389343579de277473d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:58.788556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.790882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:58.799323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 915/1841] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:58.807759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.814879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:58.964363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758f7e7-cfef-44d6-8520-3f0f26bac83c service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.012411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca47c70-a4a5-44ba-a72c-569f0b8a120e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.033983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b639c-5b36-4efb-8abb-207724f59e16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.036381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b639c-5b36-4efb-8abb-207724f59e16 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.036870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b639c-5b36-4efb-8abb-207724f59e16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.038483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b639c-5b36-4efb-8abb-207724f59e16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.044849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9d6b639c-5b36-4efb-8abb-207724f59e16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:59.074092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.075842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.076293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.076447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.090369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:59.211335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b639c-5b36-4efb-8abb-207724f59e16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.213141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b639c-5b36-4efb-8abb-207724f59e16 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.237224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 927/1842] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:59.238323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.238613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.242714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.258740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.260601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.287861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.312691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 916/1843] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:59.313919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:59.315628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:59.330203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 928/1844] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:02:59.371146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.372780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:59.379852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.381867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:59.395134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4c4f6-796e-4097-a7b4-abba6fc761f1 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.401698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a46ec-df98-4b58-9de3-51d763c0d6e3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.405953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67692e-b66d-4fb0-97ba-80307d9547ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.406966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67692e-b66d-4fb0-97ba-80307d9547ac service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.407576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67692e-b66d-4fb0-97ba-80307d9547ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.408067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67692e-b66d-4fb0-97ba-80307d9547ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.411447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.418193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.418540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.418823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] PATH_INFO: `/v3/users/8c9e5cd314134970bbd41da428f00112` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.420164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c9e5cd314134970bbd41da428f00112)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.422316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.438246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67692e-b66d-4fb0-97ba-80307d9547ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.440519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67692e-b66d-4fb0-97ba-80307d9547ac service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.457170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 929/1845] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:02:59.564911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c9e5cd314134970bbd41da428f00112'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:59.567905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c9e5cd314134970bbd41da428f00112'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:59.584683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c9e5cd314134970bbd41da428f00112'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:59.618722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c9e5cd314134970bbd41da428f00112'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:59.619528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c9e5cd314134970bbd41da428f00112'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:02:59.642154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 917/1846] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:02:59 2022] DELETE /identity/v3/users/8c9e5cd314134970bbd41da428f00112 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:59.647998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.648310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.648589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.700290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:59.704762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:59.729265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.926882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 930/1847] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:00.034281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:00.035899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:00.192009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abffdf-3cc0-4fc9-aeb9-c634cb89d2fa admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:00.197841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:00.198723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:00.199323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:00.199875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:00.215058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:00.216794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:00.233146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 918/1848] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:00.641166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:00.644459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:00.661558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dbda5-3fdb-436e-94f9-af7f531b1c77 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:00.667722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:00.668954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:00.669499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:00.669924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] PATH_INFO: `/v3/projects/9d119896b1184c82866680785604bb21` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:00.678759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d119896b1184c82866680785604bb21)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:00.682504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:00.755294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d119896b1184c82866680785604bb21', 'request_id': 'req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:00.762311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d119896b1184c82866680785604bb21', 'request_id': 'req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:00.768308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d119896b1184c82866680785604bb21', 'request_id': 'req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:00.782631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] The token cache is being invalidate because project 9d119896b1184c82866680785604bb21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:00.783073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:00.783725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef2f3a98-ff5d-4a81-9549-e1593c0fe1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:00.788632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 931/1849] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:00 2022] DELETE /identity/v3/projects/9d119896b1184c82866680785604bb21 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:00.791246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:00.805787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e82e8f-b0bb-42aa-8ba0-41fbb38359c2 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:00.884437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:00.885145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:00.885377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:00.885661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] PATH_INFO: `/v3/users/06a9ea4fb5444cfca465d176db41f27e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:00.887052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a9ea4fb5444cfca465d176db41f27e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:00.889069 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:00.946846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:01.052237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a9ea4fb5444cfca465d176db41f27e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:01.053632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a9ea4fb5444cfca465d176db41f27e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:01.062977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a9ea4fb5444cfca465d176db41f27e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:01.082048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a9ea4fb5444cfca465d176db41f27e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:01.082369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a9ea4fb5444cfca465d176db41f27e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:01.103126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 919/1850] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:00 2022] DELETE /identity/v3/users/06a9ea4fb5444cfca465d176db41f27e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:01.446992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.447273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.447769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.515267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] MFA Rules not processed for user `d5e7cd285e084f4cb1e5249489216992`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:01.517541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:01.662450 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 932/1851] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:01.663524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.665118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:01.751845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.755319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:01.783503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef69cf-7014-4b77-8b28-d1fbb7819ed2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:01.817235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:01.825856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:01.825856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.825856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.825856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.826465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:01.837483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:01.909279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 920/1852] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:01 2022] POST /identity/v3/projects => generated 373 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:01.909279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.913876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:01.938411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd000f-88f9-43c1-b568-6ed77a12753c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.005434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.006333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.006691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.007040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] PATH_INFO: `/v3/projects/d69a5089d91c4195bb096c984c55718c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.008837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69a5089d91c4195bb096c984c55718c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.011238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.030708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d25ae-708e-4e3a-9e68-d5fd971e4db3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.044336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.045628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.046065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.046344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] PATH_INFO: `/v3/projects/f8456e899a194472bf935a740f16f11c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.047813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8456e899a194472bf935a740f16f11c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.049515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.074894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.092372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69a5089d91c4195bb096c984c55718c', 'request_id': 'req-a181a49f-62a9-46cd-9744-1325fc85efd3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.105579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69a5089d91c4195bb096c984c55718c', 'request_id': 'req-a181a49f-62a9-46cd-9744-1325fc85efd3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.115618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8456e899a194472bf935a740f16f11c', 'request_id': 'req-fdbd3efd-f487-4797-b83d-549b1d33ee77'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.118943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69a5089d91c4195bb096c984c55718c', 'request_id': 'req-a181a49f-62a9-46cd-9744-1325fc85efd3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.129160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8456e899a194472bf935a740f16f11c', 'request_id': 'req-fdbd3efd-f487-4797-b83d-549b1d33ee77'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.153924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] The token cache is being invalidate because project d69a5089d91c4195bb096c984c55718c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:02.153924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.153924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.153924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 933/1853] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:01 2022] DELETE /identity/v3/projects/d69a5089d91c4195bb096c984c55718c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:02.153924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.154391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.165601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8456e899a194472bf935a740f16f11c', 'request_id': 'req-fdbd3efd-f487-4797-b83d-549b1d33ee77'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.181899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] The token cache is being invalidate because project f8456e899a194472bf935a740f16f11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:02.182198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.182961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.188596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 921/1854] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:01 2022] DELETE /identity/v3/projects/f8456e899a194472bf935a740f16f11c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:02.188596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.191099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.252018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181a49f-62a9-46cd-9744-1325fc85efd3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.261029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.280200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd3efd-f487-4797-b83d-549b1d33ee77 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.285857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.286621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.287078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.287978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.288869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.290833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.320844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 922/1855] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:02 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:02.320844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.321081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.339999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.340732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.341081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.341421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] PATH_INFO: `/v3/projects/5067fcd621254353b5d0731c1655d680` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.341782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896196db-6843-4950-9979-8db06ea4cda5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.342647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5067fcd621254353b5d0731c1655d680)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.344796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.346852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.347520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.348342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.348642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] PATH_INFO: `/v3/users/62ea47314d9840e79771bf03b45fba6a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.350847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ea47314d9840e79771bf03b45fba6a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.353141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.413771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5067fcd621254353b5d0731c1655d680', 'request_id': 'req-ff90f88f-0cd0-453b-a8c0-c6369b80da97'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.425988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5067fcd621254353b5d0731c1655d680', 'request_id': 'req-ff90f88f-0cd0-453b-a8c0-c6369b80da97'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.427037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ea47314d9840e79771bf03b45fba6a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.428577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ea47314d9840e79771bf03b45fba6a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.436365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5067fcd621254353b5d0731c1655d680', 'request_id': 'req-ff90f88f-0cd0-453b-a8c0-c6369b80da97'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.438076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ea47314d9840e79771bf03b45fba6a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.456384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] The token cache is being invalidate because project 5067fcd621254353b5d0731c1655d680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:02.456776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.457534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.460485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 934/1856] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:02 2022] DELETE /identity/v3/projects/5067fcd621254353b5d0731c1655d680 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:02.461133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.461729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ea47314d9840e79771bf03b45fba6a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.462453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ea47314d9840e79771bf03b45fba6a'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.464147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.484306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 923/1857] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:02 2022] DELETE /identity/v3/users/62ea47314d9840e79771bf03b45fba6a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:02.487520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.487520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.504947 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.570153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f88f-0cd0-453b-a8c0-c6369b80da97 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.610093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d779-e895-405b-b18c-7fc683321e24 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.618624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac67ff-f798-4c03-89e3-26772366552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.619389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac67ff-f798-4c03-89e3-26772366552d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.619770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac67ff-f798-4c03-89e3-26772366552d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.620575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac67ff-f798-4c03-89e3-26772366552d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.621262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac67ff-f798-4c03-89e3-26772366552d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.623118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac67ff-f798-4c03-89e3-26772366552d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.633718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.642504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 924/1858] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:02.642504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.642504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.642504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.667064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.667740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.668219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.668536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] PATH_INFO: `/v3/users/13ba5b4fc5c0417c99a2b946dbe72845` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.670890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ba5b4fc5c0417c99a2b946dbe72845)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.673159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.676432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:02.679027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.690167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 925/1859] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:02.691248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.692984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.705739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5172769-8ec0-43c7-9617-4ccd5f77a954 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.719940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a7f0d-b740-4760-a600-3c2d5315dd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.720646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7f0d-b740-4760-a600-3c2d5315dd7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.720927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7f0d-b740-4760-a600-3c2d5315dd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.721202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7f0d-b740-4760-a600-3c2d5315dd7b admin admin] PATH_INFO: `/v3/projects/05af09b5a814415480717d6a3facb4d7/users/46436bf8ece14df0a01d637fecc57543/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.737929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ba5b4fc5c0417c99a2b946dbe72845'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.742445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a7f0d-b740-4760-a600-3c2d5315dd7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=05af09b5a814415480717d6a3facb4d7, user_id=46436bf8ece14df0a01d637fecc57543, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.745647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a7f0d-b740-4760-a600-3c2d5315dd7b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.746390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ba5b4fc5c0417c99a2b946dbe72845'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.761664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ba5b4fc5c0417c99a2b946dbe72845'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.762399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ba5b4fc5c0417c99a2b946dbe72845'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.763524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 926/1860] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:02 2022] PUT /identity/v3/projects/05af09b5a814415480717d6a3facb4d7/users/46436bf8ece14df0a01d637fecc57543/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:02.763721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ba5b4fc5c0417c99a2b946dbe72845'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:02.767370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802410-32b3-4f4e-a68d-9e2bbab5c23d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.767705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802410-32b3-4f4e-a68d-9e2bbab5c23d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.767997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802410-32b3-4f4e-a68d-9e2bbab5c23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.785676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 935/1861] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:02 2022] DELETE /identity/v3/users/13ba5b4fc5c0417c99a2b946dbe72845 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:02.786521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.792566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.797314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-11802410-32b3-4f4e-a68d-9e2bbab5c23d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:02.799213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-11802410-32b3-4f4e-a68d-9e2bbab5c23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:02.910628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7078d-53cb-4b80-b2bd-5565dc856a24 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.918955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.918955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.919227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.919729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.920937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.922385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 927/1862] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:02.923124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.925045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11802410-32b3-4f4e-a68d-9e2bbab5c23d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.940007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11802410-32b3-4f4e-a68d-9e2bbab5c23d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.942925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 936/1863] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:02.944193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.949436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.951082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.951651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.952308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.953361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:02.953763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.955567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.956671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29005f7e-18b2-4a24-8dcb-7d8ead6798db admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.967084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.967813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.968243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.968530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] PATH_INFO: `/v3/projects/05af09b5a814415480717d6a3facb4d7/users/46436bf8ece14df0a01d637fecc57543/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.970755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05af09b5a814415480717d6a3facb4d7, user_id=46436bf8ece14df0a01d637fecc57543, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.973768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.981571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 928/1864] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:02.982606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.006334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 937/1865] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:02 2022] PUT /identity/v3/projects/05af09b5a814415480717d6a3facb4d7/users/46436bf8ece14df0a01d637fecc57543/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:03.007782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1bac5-aa8c-4a10-b94d-7dd3cf790701 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.145333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.145980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.146322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.147095 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.147791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.149705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.157349 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.192033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 929/1866] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:02 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:03.193676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.204968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5dcb5-0586-405c-863f-1fae2005940d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.292893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.310941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.312212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.313151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.313623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.314507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.316762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.355356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 938/1867] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:03 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:03.356648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.367942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ce221-0508-43f4-95bb-abe9c3a1088b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.372630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.373236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.373515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.373786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.374448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.376066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.391433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 930/1868] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:03.393118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.404670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de082046-9043-4955-a0c7-9782f697a127 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.409392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.410034 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.410314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.411110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.411326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.412864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.424108 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 939/1869] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:03.428828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.439147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8fc54-b716-4778-8c11-529b39d175a3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.445499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.446909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb60fd-245c-4771-bfb3-a0606c2dc2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.447308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.447618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb60fd-245c-4771-bfb3-a0606c2dc2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.448241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb60fd-245c-4771-bfb3-a0606c2dc2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.448523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb60fd-245c-4771-bfb3-a0606c2dc2ff admin admin] PATH_INFO: `/v3/projects/20eaa3ce22f6400da9f1cb5338e7422e/users/994a6dcaa4614c669605721bdbf3ce61/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.461585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126e7f-542b-4c72-92c3-5bb139ee2ae4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.466275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb60fd-245c-4771-bfb3-a0606c2dc2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=20eaa3ce22f6400da9f1cb5338e7422e, user_id=994a6dcaa4614c669605721bdbf3ce61, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.467297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.467962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.468237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.468506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] PATH_INFO: `/v3/projects/651dd53750904bb996c87e4677ed31be` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.469627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb60fd-245c-4771-bfb3-a0606c2dc2ff admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.469728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=651dd53750904bb996c87e4677ed31be)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.471473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.487759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 931/1870] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:03 2022] PUT /identity/v3/projects/20eaa3ce22f6400da9f1cb5338e7422e/users/994a6dcaa4614c669605721bdbf3ce61/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:03.499384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.499690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.499958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.536146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] MFA Rules not processed for user `994a6dcaa4614c669605721bdbf3ce61`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:03.537991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.550334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651dd53750904bb996c87e4677ed31be', 'request_id': 'req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:03.560686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651dd53750904bb996c87e4677ed31be', 'request_id': 'req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:03.569755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651dd53750904bb996c87e4677ed31be', 'request_id': 'req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:03.583931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] The token cache is being invalidate because project 651dd53750904bb996c87e4677ed31be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:03.583931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:03.584653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c56511e1-b1e0-4ab8-b044-d9be7b2cf6b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:03.587212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 940/1871] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:03 2022] DELETE /identity/v3/projects/651dd53750904bb996c87e4677ed31be => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:03.592586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.592973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.593359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.617905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:03.619032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.680573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 932/1872] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:03.681562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.683308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.704747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 941/1873] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:03.704747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.704747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.722650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cff3af-230e-4510-9e9b-6a5b5ba3c230 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.735058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8f82a-3ef7-4d03-9cdb-e0e72027a6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.735704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f82a-3ef7-4d03-9cdb-e0e72027a6a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.736079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f82a-3ef7-4d03-9cdb-e0e72027a6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.736475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f82a-3ef7-4d03-9cdb-e0e72027a6a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.737213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8f82a-3ef7-4d03-9cdb-e0e72027a6a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.739346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8f82a-3ef7-4d03-9cdb-e0e72027a6a1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.782233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 942/1874] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:03 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:03.784333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c6e44-14c1-43d9-b975-21469858ef97 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.784746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c6e44-14c1-43d9-b975-21469858ef97 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.785118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c6e44-14c1-43d9-b975-21469858ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.785224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baec2c-006c-415a-96e9-0d09e8707996 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.810164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ee2c6e44-14c1-43d9-b975-21469858ef97 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:03.813461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee2c6e44-14c1-43d9-b975-21469858ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.852570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.878297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.900838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.901622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.902022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.902605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] PATH_INFO: `/v3/projects/92da77fbd4f244a4ac70be3a1b5bb2d5` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.904262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92da77fbd4f244a4ac70be3a1b5bb2d5)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.907040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.911423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 943/1875] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:03.911423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c6e44-14c1-43d9-b975-21469858ef97 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.925521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c6e44-14c1-43d9-b975-21469858ef97 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.931950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.932724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.933247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.933585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.934307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.937289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.969878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 944/1876] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:03 2022] POST /identity/v3/users => generated 299 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:03.975066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.991586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b471e5-f42e-4076-b59c-3deae93a2036 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.997960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.998732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.999046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.999537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.000204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.000580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.002763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.019029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 945/1877] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.020117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.032419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92da77fbd4f244a4ac70be3a1b5bb2d5', 'request_id': 'req-2681ac14-859c-4ffc-a339-de55a5cc337b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:04.032579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4adb33-367f-4234-b6d8-fad27415d7f3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.042848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92da77fbd4f244a4ac70be3a1b5bb2d5', 'request_id': 'req-2681ac14-859c-4ffc-a339-de55a5cc337b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:04.058733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92da77fbd4f244a4ac70be3a1b5bb2d5', 'request_id': 'req-2681ac14-859c-4ffc-a339-de55a5cc337b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:04.074334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] The token cache is being invalidate because project 92da77fbd4f244a4ac70be3a1b5bb2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:04.074792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:04.076640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:04.078492 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 933/1878] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:03 2022] DELETE /identity/v3/projects/92da77fbd4f244a4ac70be3a1b5bb2d5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:04.079953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.081923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.122431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85c7b9-0ad2-4070-8cb6-3ca2228cda6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.123283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85c7b9-0ad2-4070-8cb6-3ca2228cda6a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.123860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85c7b9-0ad2-4070-8cb6-3ca2228cda6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.124354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85c7b9-0ad2-4070-8cb6-3ca2228cda6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.125296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85c7b9-0ad2-4070-8cb6-3ca2228cda6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.127442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85c7b9-0ad2-4070-8cb6-3ca2228cda6a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.144321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 946/1879] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:04.146294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.146688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.147062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.172750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ac14-859c-4ffc-a339-de55a5cc337b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.175202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:04.176824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.181202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cdc45-8466-413c-ad5d-e854f94d479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.182374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cdc45-8466-413c-ad5d-e854f94d479d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.182374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cdc45-8466-413c-ad5d-e854f94d479d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.182590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cdc45-8466-413c-ad5d-e854f94d479d admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.183205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cdc45-8466-413c-ad5d-e854f94d479d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.185252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cdc45-8466-413c-ad5d-e854f94d479d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.187544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 947/1880] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:04.188717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.190398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.204994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288844-6582-48e7-bbfd-d2890412e406 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.211671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.212659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.213195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.213562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] PATH_INFO: `/v3/projects/17a3c1805a4a493dbf65d4fe0df43ea3/users/502eb8757d7a4b72b3882c2fd6b20822/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.223620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 934/1881] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:04.225434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871b2f-0c75-4683-9289-1ee13edaa901 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.226386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871b2f-0c75-4683-9289-1ee13edaa901 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.226894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8871b2f-0c75-4683-9289-1ee13edaa901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.244265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a3c1805a4a493dbf65d4fe0df43ea3, user_id=502eb8757d7a4b72b3882c2fd6b20822, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.249175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.268546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f8871b2f-0c75-4683-9289-1ee13edaa901 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:04.274058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f8871b2f-0c75-4683-9289-1ee13edaa901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.307822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 948/1882] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:04 2022] PUT /identity/v3/projects/17a3c1805a4a493dbf65d4fe0df43ea3/users/502eb8757d7a4b72b3882c2fd6b20822/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:04.307822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.417377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de18b9-4f71-4bfa-bdbf-941fb13e9edf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.423629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.424248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.424536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.424813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.425644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.428490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.433219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 935/1883] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:04.434344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8871b2f-0c75-4683-9289-1ee13edaa901 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.445302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8871b2f-0c75-4683-9289-1ee13edaa901 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.446178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.447911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.456134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.457750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.458250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.459725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.461128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.463712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 949/1884] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:04.465209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.470184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.484863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 936/1885] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:04.484863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.490376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c3563-8145-41f3-ba50-746051c6230a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.497739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91215-4ac1-4fb1-abd1-ad84b7d48368 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.501215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.502236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.502754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.503168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.503516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.504199 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.504467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.505030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.505488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.506521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.506928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.508372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.509339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.523652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 937/1886] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:04.524703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.527401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 950/1887] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.528676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.536599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f50ee-1476-4a0e-8d22-13bc576ebfda admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.542517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f8fd3-16a2-4095-a514-8c1d1c75e063 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.543327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.544278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.545870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.546409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] PATH_INFO: `/v3/projects/17a3c1805a4a493dbf65d4fe0df43ea3/users/502eb8757d7a4b72b3882c2fd6b20822/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.548454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e2958-1838-4676-af6e-4da9846503d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.549219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e2958-1838-4676-af6e-4da9846503d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.549334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a3c1805a4a493dbf65d4fe0df43ea3, user_id=502eb8757d7a4b72b3882c2fd6b20822, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.549855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e2958-1838-4676-af6e-4da9846503d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.550282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e2958-1838-4676-af6e-4da9846503d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.551329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e2958-1838-4676-af6e-4da9846503d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.553512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.560853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e2958-1838-4676-af6e-4da9846503d5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.581149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 938/1888] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:04 2022] PUT /identity/v3/projects/17a3c1805a4a493dbf65d4fe0df43ea3/users/502eb8757d7a4b72b3882c2fd6b20822/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:04.581149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.581584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 951/1889] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:04.583248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e72109-70f6-41d5-93f5-4f9fa6c277f0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.583450 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e72109-70f6-41d5-93f5-4f9fa6c277f0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.583801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e72109-70f6-41d5-93f5-4f9fa6c277f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.620217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-42e72109-70f6-41d5-93f5-4f9fa6c277f0 None None] MFA Rules not processed for user `502eb8757d7a4b72b3882c2fd6b20822`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:04.623076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-42e72109-70f6-41d5-93f5-4f9fa6c277f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.698099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf26eb-8987-44e8-90de-0ba2855f5465 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.704345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.705397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.705892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.707112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.708155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.717187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.721570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 952/1890] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:04.721570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e72109-70f6-41d5-93f5-4f9fa6c277f0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.739156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e72109-70f6-41d5-93f5-4f9fa6c277f0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.750396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.751261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.751645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.752194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] PATH_INFO: `/v3/projects/ca9594c6dd5748dc8bedf0874c5ed74f/users/e3c801535b5c49e0bd72451dd791ba42/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.773023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9594c6dd5748dc8bedf0874c5ed74f, user_id=e3c801535b5c49e0bd72451dd791ba42, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.777690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.784703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 939/1891] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/projects => generated 399 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:04.785847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.791683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.798840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 953/1892] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:04 2022] PUT /identity/v3/projects/ca9594c6dd5748dc8bedf0874c5ed74f/users/e3c801535b5c49e0bd72451dd791ba42/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:04.800044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.887117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10269f-d111-4531-81ad-cdaf823ae3ed admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.892989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658c2ac-cffa-4c26-b363-a6d8194c4aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.893931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c2ac-cffa-4c26-b363-a6d8194c4aed admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.894411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c2ac-cffa-4c26-b363-a6d8194c4aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.894835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c2ac-cffa-4c26-b363-a6d8194c4aed admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.895704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658c2ac-cffa-4c26-b363-a6d8194c4aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.897942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658c2ac-cffa-4c26-b363-a6d8194c4aed admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.903209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c121-c9fd-4b25-b466-0421733095e2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.924218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 954/1893] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:04.926749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe23ac-2b85-4746-b2ee-42c219a128ca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.928224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe23ac-2b85-4746-b2ee-42c219a128ca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.928656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe23ac-2b85-4746-b2ee-42c219a128ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.960165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bfbe23ac-2b85-4746-b2ee-42c219a128ca None None] MFA Rules not processed for user `e3c801535b5c49e0bd72451dd791ba42`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:04.961989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.962490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bfbe23ac-2b85-4746-b2ee-42c219a128ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.962671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.962894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.963160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.966083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:04.977369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.994761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.041588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 955/1894] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:05.042088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.044271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.045943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe23ac-2b85-4746-b2ee-42c219a128ca None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.056824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe23ac-2b85-4746-b2ee-42c219a128ca None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.058935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 940/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:05.058935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.070561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eff95-2304-40b5-8725-8e8e910e5984 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.076669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.077480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.077867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.078243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.079907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.082774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.084252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.085011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.085173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.085442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.086096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.087873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.125955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 956/1896] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:05.126172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 941/1897] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:05 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:05.129332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.134327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.158005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0caacb-c3d3-4359-bea2-7b52c5cb3ba8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.240617 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78119924-91f2-4c55-97e7-e0183031f68e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.245792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.246933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.247417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.248214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.249057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.251059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.269564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 942/1898] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:05.270934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.271834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.272476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.273492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.273826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.274480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.280192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.285564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8fe31-3c6f-472d-8ebc-307fe6ee3aff admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.294807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51370c88-6048-416b-b73b-ad9e4e5ecf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.296308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370c88-6048-416b-b73b-ad9e4e5ecf06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.296796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370c88-6048-416b-b73b-ad9e4e5ecf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.297729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51370c88-6048-416b-b73b-ad9e4e5ecf06 admin admin] PATH_INFO: `/v3/projects/1bf6a3e9f87b48a0ac9140716b6085ac/users/f47c25126c644019b3f55392366f6797/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.325718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51370c88-6048-416b-b73b-ad9e4e5ecf06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf6a3e9f87b48a0ac9140716b6085ac, user_id=f47c25126c644019b3f55392366f6797, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.326700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 957/1899] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:05 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:05.330751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.331903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51370c88-6048-416b-b73b-ad9e4e5ecf06 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.348217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c4b21-29d3-4500-9068-37ce3f0ff75d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.356009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.356640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.356923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.357248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.358049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.359963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.376695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 958/1900] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:05.378668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 943/1901] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:05 2022] PUT /identity/v3/projects/1bf6a3e9f87b48a0ac9140716b6085ac/users/f47c25126c644019b3f55392366f6797/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:05.378790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.407677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12219f-fc31-4274-99eb-c2a3421c6972 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.408054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12219f-fc31-4274-99eb-c2a3421c6972 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.408336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12219f-fc31-4274-99eb-c2a3421c6972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.413388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556daf8-9727-449e-ab61-41406bfef699 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.449947 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fe12219f-fc31-4274-99eb-c2a3421c6972 None None] MFA Rules not processed for user `f47c25126c644019b3f55392366f6797`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:05.451988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fe12219f-fc31-4274-99eb-c2a3421c6972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:05.564343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.567127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.567404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.573656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] PATH_INFO: `/v3/projects/b846343a66e04936a892d965b99e3bea/users/44e67b2155b94b1f953a76cead7951b8/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.575031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 944/1902] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:05.576668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12219f-fc31-4274-99eb-c2a3421c6972 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.608806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b846343a66e04936a892d965b99e3bea, user_id=44e67b2155b94b1f953a76cead7951b8, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.611983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.634763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.644623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 959/1903] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:05 2022] PUT /identity/v3/projects/b846343a66e04936a892d965b99e3bea/users/44e67b2155b94b1f953a76cead7951b8/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:05.647172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12219f-fc31-4274-99eb-c2a3421c6972 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.659817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.683321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.793524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.794433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.796155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.796453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.797909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.801701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.830271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0ed38-f3f6-4e9e-b9eb-ae3d3739f16c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.839886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.840852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.841336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.841823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.842493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.844636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.857604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 945/1904] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:05 2022] POST /identity/v3/projects => generated 399 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:05.859872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 960/1905] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:05.860330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.861445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.874391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5a4a-fd39-4a1f-9a22-9d1aed9e8979 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.957217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530243-094c-4508-94b6-97949d28aed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.957934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530243-094c-4508-94b6-97949d28aed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.958221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530243-094c-4508-94b6-97949d28aed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.959282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530243-094c-4508-94b6-97949d28aed1 admin admin] PATH_INFO: `/v3/projects/b846343a66e04936a892d965b99e3bea/users/44e67b2155b94b1f953a76cead7951b8/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.961140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452a58a-f0c4-4f57-b6f6-a2716249df73 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.963220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530243-094c-4508-94b6-97949d28aed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b846343a66e04936a892d965b99e3bea, user_id=44e67b2155b94b1f953a76cead7951b8, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.967189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530243-094c-4508-94b6-97949d28aed1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.973970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.974786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.975074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.975348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.976016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.977982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.997033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 961/1906] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:05 2022] PUT /identity/v3/projects/b846343a66e04936a892d965b99e3bea/users/44e67b2155b94b1f953a76cead7951b8/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:06.009235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6862b89-f454-460f-a09b-178cbf29b980 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.009542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6862b89-f454-460f-a09b-178cbf29b980 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.009820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6862b89-f454-460f-a09b-178cbf29b980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.023201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 946/1907] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:05 2022] POST /identity/v3/users => generated 306 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:06.028036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.039326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff044722-11b4-4e75-90fc-a8c88cb0a755 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.051366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d6862b89-f454-460f-a09b-178cbf29b980 None None] MFA Rules not processed for user `44e67b2155b94b1f953a76cead7951b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:06.053181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d6862b89-f454-460f-a09b-178cbf29b980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:06.113264 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.113997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.114276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.114622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.115861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.117691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.136389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 962/1908] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:06.143601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 947/1909] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:06.149202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6862b89-f454-460f-a09b-178cbf29b980 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.150542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.156411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:06.172426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6862b89-f454-460f-a09b-178cbf29b980 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.181871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.183237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.183928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.184210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] PATH_INFO: `/v3/projects/a31774ee35e04fbfb41f11e42c21bfbb/users/295d6c9410c84853a87258a9b7e72ade/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.203299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.215953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31774ee35e04fbfb41f11e42c21bfbb, user_id=295d6c9410c84853a87258a9b7e72ade, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.221779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.226654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.248708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 963/1910] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:06 2022] PUT /identity/v3/projects/a31774ee35e04fbfb41f11e42c21bfbb/users/295d6c9410c84853a87258a9b7e72ade/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:06.255304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.262988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429e276-27b0-4845-889f-25132b09ed22 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.332882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.334364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.334695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.334973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.358661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8411-45c1-4d62-a2a8-1c357c4f1eea admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.364156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.365710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.365998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.366275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.367154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.368681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.382198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 964/1911] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:06.383322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.393654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118d8ff-6925-4db4-a9aa-599e0787d792 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.398592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe95e28-5a3e-416e-9f61-d432dfdced43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.399157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95e28-5a3e-416e-9f61-d432dfdced43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.399446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95e28-5a3e-416e-9f61-d432dfdced43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.399880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95e28-5a3e-416e-9f61-d432dfdced43 admin admin] PATH_INFO: `/v3/projects/a31774ee35e04fbfb41f11e42c21bfbb/users/295d6c9410c84853a87258a9b7e72ade/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.401807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe95e28-5a3e-416e-9f61-d432dfdced43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31774ee35e04fbfb41f11e42c21bfbb, user_id=295d6c9410c84853a87258a9b7e72ade, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.404497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe95e28-5a3e-416e-9f61-d432dfdced43 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.413379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.415080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.416682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 965/1912] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:06 2022] PUT /identity/v3/projects/a31774ee35e04fbfb41f11e42c21bfbb/users/295d6c9410c84853a87258a9b7e72ade/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:06.430917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc968c4-4507-41e6-b2c0-404d2a1b93f5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.431307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc968c4-4507-41e6-b2c0-404d2a1b93f5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.431754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc968c4-4507-41e6-b2c0-404d2a1b93f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.453042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 948/1913] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:06.468684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6cc968c4-4507-41e6-b2c0-404d2a1b93f5 None None] MFA Rules not processed for user `295d6c9410c84853a87258a9b7e72ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:06.470062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6cc968c4-4507-41e6-b2c0-404d2a1b93f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:06.501747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.517684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9f9cb-1fa2-47c4-9a60-beece5a5609d service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.619217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 966/1914] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:06.628340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc968c4-4507-41e6-b2c0-404d2a1b93f5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.674532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.676185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.676478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.676754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] PATH_INFO: `/v3/users/e3c801535b5c49e0bd72451dd791ba42` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.680594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c801535b5c49e0bd72451dd791ba42)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.683595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.726974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc968c4-4507-41e6-b2c0-404d2a1b93f5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.732948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.734797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f46c77-0969-4bd0-9a5f-415b4a517d45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.746713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f46c77-0969-4bd0-9a5f-415b4a517d45 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.747002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f46c77-0969-4bd0-9a5f-415b4a517d45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.747286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f46c77-0969-4bd0-9a5f-415b4a517d45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.765231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f46c77-0969-4bd0-9a5f-415b4a517d45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.767611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c801535b5c49e0bd72451dd791ba42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:06.768130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f46c77-0969-4bd0-9a5f-415b4a517d45 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.768458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c801535b5c49e0bd72451dd791ba42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:06.778497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c801535b5c49e0bd72451dd791ba42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:06.784257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 967/1915] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:06.799045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.799045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.799045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.800288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c801535b5c49e0bd72451dd791ba42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:06.800288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3cae44b4-6908-432a-b63c-dd6ddaf5c3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c801535b5c49e0bd72451dd791ba42'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:06.804726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.817194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 949/1916] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:06 2022] DELETE /identity/v3/users/e3c801535b5c49e0bd72451dd791ba42 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:06.823428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.823738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.824011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.852950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] MFA Rules not processed for user `46436bf8ece14df0a01d637fecc57543`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:06.854839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:06.855063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:06.856277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:06.958050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 968/1917] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:06.958050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.958050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:06.959686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 950/1918] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:06.971055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.973670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:07.028317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66865d8-5625-4d28-9373-f4bb8d2cd6ac None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:07.036156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:07.036687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.037730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.037993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.040301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:07.048786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa3a2-d71c-4ffd-9643-704632fdfc7f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:07.054481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:07.055107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.056043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.056043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.070160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:07.072704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:07.094453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 951/1919] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:07.152934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:07.155690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:07.175410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 969/1920] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:07.225081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:07.226657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:07.244908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4e577-5c4f-4528-9e74-ab9c7103be19 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:07.250379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:07.251719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.252047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.252408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.254693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:07.362542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:07.414754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:07.416256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:07.430299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 952/1921] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:08.165839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.167293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:08.182039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38da1a7-5300-4111-98f8-6e725b52929b service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.187315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.188052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.188469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.188810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] PATH_INFO: `/v3/projects/ca9594c6dd5748dc8bedf0874c5ed74f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.190217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9594c6dd5748dc8bedf0874c5ed74f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:08.192505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:08.248731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:08.258092 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9594c6dd5748dc8bedf0874c5ed74f', 'request_id': 'req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.264217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9594c6dd5748dc8bedf0874c5ed74f', 'request_id': 'req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.269724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9594c6dd5748dc8bedf0874c5ed74f', 'request_id': 'req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.281954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] The token cache is being invalidate because project ca9594c6dd5748dc8bedf0874c5ed74f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:08.282244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.282947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.284478 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 970/1922] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:08 2022] DELETE /identity/v3/projects/ca9594c6dd5748dc8bedf0874c5ed74f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:08.288450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.299019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2330-33ad-4bca-8b89-ff550cf299b6 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.359568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.360112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.360393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.360664 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] PATH_INFO: `/v3/users/44e67b2155b94b1f953a76cead7951b8` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.361777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e67b2155b94b1f953a76cead7951b8)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:08.364291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:08.412371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e67b2155b94b1f953a76cead7951b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.413553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e67b2155b94b1f953a76cead7951b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.420578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e67b2155b94b1f953a76cead7951b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.433338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e67b2155b94b1f953a76cead7951b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.434009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-92ad5874-c5d3-439d-bcd4-f026599a1974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e67b2155b94b1f953a76cead7951b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.447517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 953/1923] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:08 2022] DELETE /identity/v3/users/44e67b2155b94b1f953a76cead7951b8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:08.712958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.714900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:08.732863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00b6ed-bf3b-4a25-8a95-2c423ab46c6b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.796867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.797512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.797988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.798266 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] PATH_INFO: `/v3/projects/b846343a66e04936a892d965b99e3bea` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.799403 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b846343a66e04936a892d965b99e3bea)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:08.801704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:08.858260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b846343a66e04936a892d965b99e3bea', 'request_id': 'req-16439575-cc21-4843-a036-5193dd94b904'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.866690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b846343a66e04936a892d965b99e3bea', 'request_id': 'req-16439575-cc21-4843-a036-5193dd94b904'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.875185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b846343a66e04936a892d965b99e3bea', 'request_id': 'req-16439575-cc21-4843-a036-5193dd94b904'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.892229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] The token cache is being invalidate because project b846343a66e04936a892d965b99e3bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:08.892532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.893234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16439575-cc21-4843-a036-5193dd94b904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:08.895334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 971/1924] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:08 2022] DELETE /identity/v3/projects/b846343a66e04936a892d965b99e3bea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:08.900390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec762a4-3be7-425a-b54c-1705521b780a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.900684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec762a4-3be7-425a-b54c-1705521b780a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.900963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec762a4-3be7-425a-b54c-1705521b780a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.960291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5ec762a4-3be7-425a-b54c-1705521b780a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:08.961611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5ec762a4-3be7-425a-b54c-1705521b780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:09.131264 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 954/1925] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:09.138546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f57075-502e-451b-9c51-00bad71360e1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.138832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f57075-502e-451b-9c51-00bad71360e1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.139164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f57075-502e-451b-9c51-00bad71360e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.172325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b1f57075-502e-451b-9c51-00bad71360e1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:09.174634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b1f57075-502e-451b-9c51-00bad71360e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:09.188944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 972/1926] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:09.193002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec762a4-3be7-425a-b54c-1705521b780a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.198094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.208392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec762a4-3be7-425a-b54c-1705521b780a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.219705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.220337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.220673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.220944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.221603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:09.221906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.223595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.250725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 955/1927] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:09.254321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f57075-502e-451b-9c51-00bad71360e1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.273114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f57075-502e-451b-9c51-00bad71360e1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.289320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.290402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.290734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.291097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.291889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.294126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.296533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.298289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:09.366062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 973/1928] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:09.370809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.461047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e1ed4-5b33-433b-b7fb-4fefd8596cb8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.481221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.539920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954ac0-6a34-4145-988e-30cfa9251de5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.546527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.547379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.548054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.548563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.549683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.552162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.596058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 974/1929] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:09.597589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.617175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f0e7-91df-43d6-a25e-bb50a3b8a9be admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.631529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.631529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.631529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.631529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.631960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.632044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ee35b-74c7-4707-ad72-33e5920f0866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.632151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.632243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ee35b-74c7-4707-ad72-33e5920f0866 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.632243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ee35b-74c7-4707-ad72-33e5920f0866 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.632243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ee35b-74c7-4707-ad72-33e5920f0866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.632243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-599ee35b-74c7-4707-ad72-33e5920f0866 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:09.655681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 975/1930] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:09.658831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.673772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09093e36-2b3c-455c-a640-e79990232468 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.682265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.683237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.683727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.684137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.685129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.687978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.701799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 976/1931] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:09.706256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.721655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ef022-47fb-492c-81ef-37cc61873acb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.726707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8f1c0-5e82-48bb-a5fb-95465a7492cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.727298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ee35b-74c7-4707-ad72-33e5920f0866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.727512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f1c0-5e82-48bb-a5fb-95465a7492cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.727837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f1c0-5e82-48bb-a5fb-95465a7492cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.728250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f1c0-5e82-48bb-a5fb-95465a7492cb admin admin] PATH_INFO: `/v3/projects/1a0603edf6f94eb788dbd8567826cf9a/users/b5610cac368348e396395ccd39ae4eee/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.728974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ee35b-74c7-4707-ad72-33e5920f0866 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.743747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 956/1932] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:09.749859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8f1c0-5e82-48bb-a5fb-95465a7492cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0603edf6f94eb788dbd8567826cf9a, user_id=b5610cac368348e396395ccd39ae4eee, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.752588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8f1c0-5e82-48bb-a5fb-95465a7492cb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.769231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 977/1933] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:09 2022] PUT /identity/v3/projects/1a0603edf6f94eb788dbd8567826cf9a/users/b5610cac368348e396395ccd39ae4eee/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:09.780483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c810405-e89b-43c7-a93b-4c729ccc83e7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.781019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c810405-e89b-43c7-a93b-4c729ccc83e7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.781365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c810405-e89b-43c7-a93b-4c729ccc83e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.783710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826687ac-1f61-489d-a573-ba5db7d9cd73 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.783832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826687ac-1f61-489d-a573-ba5db7d9cd73 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.784105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826687ac-1f61-489d-a573-ba5db7d9cd73 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.786292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 978/1934] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:09.828027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6c810405-e89b-43c7-a93b-4c729ccc83e7 None None] MFA Rules not processed for user `b5610cac368348e396395ccd39ae4eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:09.829246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6c810405-e89b-43c7-a93b-4c729ccc83e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:09.831560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.832610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.833473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.889326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:09.891287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:09.893112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.930597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 957/1935] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:09.936724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c810405-e89b-43c7-a93b-4c729ccc83e7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.020445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 979/1936] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:10.040863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.044233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:10.053509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c810405-e89b-43c7-a93b-4c729ccc83e7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.058655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.059290 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.059599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.059871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.060516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.062541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.068853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38a7cd-8893-4099-9302-d64e09650ae7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.077526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.078501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.078855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.079201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.094848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.096824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.104475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 958/1937] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/projects => generated 393 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:10.108327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.110234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 980/1938] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:10.217784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316e0a2-f82e-4ae1-b05c-685f53032cb8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.235599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.236251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.237293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.237921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.243655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.247088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.282528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 959/1939] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:10 2022] POST /identity/v3/users => generated 303 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:10.286188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.298418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6ad8c-7e65-4366-bc6e-2bf728f4d9e0 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.304085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.304707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.305109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.305394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.306174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.308492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.328685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 981/1940] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:10.333222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.351938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb114e-439d-4185-a4f7-c06d84a1c9b4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.383517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.384318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.384598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.384921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] PATH_INFO: `/v3/projects/68933080acc94324bf99bf775fc7bdb9/users/076a1ca4d8ff40b99030f3046acbdfbc/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.408065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68933080acc94324bf99bf775fc7bdb9, user_id=076a1ca4d8ff40b99030f3046acbdfbc, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.411162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.429407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 960/1941] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:10 2022] PUT /identity/v3/projects/68933080acc94324bf99bf775fc7bdb9/users/076a1ca4d8ff40b99030f3046acbdfbc/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:10.433449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.448561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e288-fb34-4e4c-a8e0-dcad9f3c335e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.501304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.549361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eab64-fae7-4536-a99a-dac417fdb6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.550247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eab64-fae7-4536-a99a-dac417fdb6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.550570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eab64-fae7-4536-a99a-dac417fdb6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.551137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eab64-fae7-4536-a99a-dac417fdb6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.551998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eab64-fae7-4536-a99a-dac417fdb6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.554152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eab64-fae7-4536-a99a-dac417fdb6a7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.565905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 982/1942] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:10.569867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.584919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06b315-bf3c-45d9-b503-b8d4c7334487 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.592751 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.593490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.593773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.594050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] PATH_INFO: `/v3/projects/68933080acc94324bf99bf775fc7bdb9/users/076a1ca4d8ff40b99030f3046acbdfbc/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.597702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68933080acc94324bf99bf775fc7bdb9, user_id=076a1ca4d8ff40b99030f3046acbdfbc, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.602415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.618983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 961/1943] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:10 2022] PUT /identity/v3/projects/68933080acc94324bf99bf775fc7bdb9/users/076a1ca4d8ff40b99030f3046acbdfbc/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:10.632105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.632786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.633342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.718884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] MFA Rules not processed for user `076a1ca4d8ff40b99030f3046acbdfbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:10.720612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:10.768672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.840453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.876332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 983/1944] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:10 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:10.882210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4c5a5-4bb1-4850-9646-f6dab87c1253 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.888144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.888846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.889782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.889782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.891411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:10.891940 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.893491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:10.906460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e829fa-d863-46c5-a653-261d0513fcb8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.910687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.911335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.911643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.912442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.926814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.932502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.943610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 984/1945] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:10.964800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.966216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.976337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 962/1946] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:11.042352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.051245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd048f2-b6e9-4556-9f0c-fe9c29fadcba service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.054993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a207b7-e047-47cb-ba62-781343caa19b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.055627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a207b7-e047-47cb-ba62-781343caa19b service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.055958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a207b7-e047-47cb-ba62-781343caa19b service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.056192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a207b7-e047-47cb-ba62-781343caa19b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.115741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a207b7-e047-47cb-ba62-781343caa19b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.117384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a207b7-e047-47cb-ba62-781343caa19b service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.126706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 985/1947] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:12.457140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.459066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:12.583475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad93919-f645-45fb-8e98-ded1fe788081 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.589382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.590131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.590409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.591290 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] PATH_INFO: `/v3/users/f47c25126c644019b3f55392366f6797` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.592640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47c25126c644019b3f55392366f6797)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.595034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.659964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47c25126c644019b3f55392366f6797'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:12.660649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47c25126c644019b3f55392366f6797'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:12.668534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47c25126c644019b3f55392366f6797'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:12.682885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47c25126c644019b3f55392366f6797'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:12.683713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47c25126c644019b3f55392366f6797'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:12.685702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.703843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 963/1948] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:12 2022] DELETE /identity/v3/users/f47c25126c644019b3f55392366f6797 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:12.708055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.709064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.709257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.732115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:12.733340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:12.795283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.823641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 986/1949] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:12.831088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.832558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:12.921294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d48a7-af98-40c3-bf5e-ca282cd15207 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.925439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.926490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.926772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.927050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.938227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.939815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.956358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 964/1950] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:13.595074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.596829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:13.618665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a80093-384c-4853-b360-50e084a2d70a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.627136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.628142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.628489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.628860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] PATH_INFO: `/v3/projects/1bf6a3e9f87b48a0ac9140716b6085ac` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.630823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf6a3e9f87b48a0ac9140716b6085ac)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.633451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.719544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf6a3e9f87b48a0ac9140716b6085ac', 'request_id': 'req-ce5e1d66-5ed9-48ec-ab12-978035b85466'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.732924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf6a3e9f87b48a0ac9140716b6085ac', 'request_id': 'req-ce5e1d66-5ed9-48ec-ab12-978035b85466'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.742668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf6a3e9f87b48a0ac9140716b6085ac', 'request_id': 'req-ce5e1d66-5ed9-48ec-ab12-978035b85466'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.756164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] The token cache is being invalidate because project 1bf6a3e9f87b48a0ac9140716b6085ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:13.756709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.757334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.758822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 987/1951] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:13 2022] DELETE /identity/v3/projects/1bf6a3e9f87b48a0ac9140716b6085ac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:13.762960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.772334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b592f-f635-4335-b8f6-882de03805a5 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.851221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.852032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.852319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.852597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] PATH_INFO: `/v3/users/295d6c9410c84853a87258a9b7e72ade` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.853790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=295d6c9410c84853a87258a9b7e72ade)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.855784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.931282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '295d6c9410c84853a87258a9b7e72ade'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.932250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '295d6c9410c84853a87258a9b7e72ade'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.940769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '295d6c9410c84853a87258a9b7e72ade'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.954843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '295d6c9410c84853a87258a9b7e72ade'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.955616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-31aa15f8-9c6d-4288-a33d-22ae6edb38c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '295d6c9410c84853a87258a9b7e72ade'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:13.970116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 965/1952] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:13 2022] DELETE /identity/v3/users/295d6c9410c84853a87258a9b7e72ade => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:14.266784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.268636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:14.292234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1d66-5ed9-48ec-ab12-978035b85466 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.378933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.379600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.380767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.381067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] PATH_INFO: `/v3/projects/a31774ee35e04fbfb41f11e42c21bfbb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.383307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31774ee35e04fbfb41f11e42c21bfbb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.386093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.437010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31774ee35e04fbfb41f11e42c21bfbb', 'request_id': 'req-18303193-e864-48f8-b287-aec60f614f3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:14.443093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31774ee35e04fbfb41f11e42c21bfbb', 'request_id': 'req-18303193-e864-48f8-b287-aec60f614f3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:14.448687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31774ee35e04fbfb41f11e42c21bfbb', 'request_id': 'req-18303193-e864-48f8-b287-aec60f614f3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:14.458528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.463245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] The token cache is being invalidate because project a31774ee35e04fbfb41f11e42c21bfbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:14.463857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:14.464885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-18303193-e864-48f8-b287-aec60f614f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:14.466730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 988/1953] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:14 2022] DELETE /identity/v3/projects/a31774ee35e04fbfb41f11e42c21bfbb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:14.471605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a915421-56c7-4d9d-b0a1-cc5304d0d2fa None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.471896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a915421-56c7-4d9d-b0a1-cc5304d0d2fa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.472172 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a915421-56c7-4d9d-b0a1-cc5304d0d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.497420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4a915421-56c7-4d9d-b0a1-cc5304d0d2fa None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:14.498747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4a915421-56c7-4d9d-b0a1-cc5304d0d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:14.519952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.573429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 966/1954] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:14.581943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba221-c943-4fd5-b194-9a2b4b21b5e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.582494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba221-c943-4fd5-b194-9a2b4b21b5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.583014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba221-c943-4fd5-b194-9a2b4b21b5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.611400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-857ba221-c943-4fd5-b194-9a2b4b21b5e9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:14.614405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-857ba221-c943-4fd5-b194-9a2b4b21b5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:14.624702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 989/1955] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:14.628530 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a915421-56c7-4d9d-b0a1-cc5304d0d2fa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.642275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a915421-56c7-4d9d-b0a1-cc5304d0d2fa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.653708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.655576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.655981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.656337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.657079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:14.657454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.659125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.673194 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 967/1956] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:14.678162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba221-c943-4fd5-b194-9a2b4b21b5e9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.688450 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba221-c943-4fd5-b194-9a2b4b21b5e9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.693903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.694511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.694906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.695357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.695841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.697776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.738013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 990/1957] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:14 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:14.742811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.752723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c82733-c519-42c2-a6dc-ceab9f6bd7a0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.817166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.817833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.818066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.818503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.819193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.821180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.845220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 968/1958] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:14 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:14.850425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.861143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8818c-3ed4-417e-a894-640dcf651624 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.865812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.866433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.866710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.866982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.867693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.869349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.880054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 991/1959] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:14.884305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.894682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d22af0-bfc1-4fc7-ac03-3d113099ecbd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.900034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50251-6e88-450d-ace6-4591180f8f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.900659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50251-6e88-450d-ace6-4591180f8f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.900941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50251-6e88-450d-ace6-4591180f8f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.901217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50251-6e88-450d-ace6-4591180f8f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.901868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f50251-6e88-450d-ace6-4591180f8f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.903511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f50251-6e88-450d-ace6-4591180f8f2d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.918817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 969/1960] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:14.920185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.930751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0e40-a2c7-48a6-afa8-5ab5702790a6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.935456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.936154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.936441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.936710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] PATH_INFO: `/v3/projects/08d48aaadfe34ed6a30d1f90ffbf11fa/users/2141190cb28a48198c8b4bc8ffc2bb2e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.954472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d48aaadfe34ed6a30d1f90ffbf11fa, user_id=2141190cb28a48198c8b4bc8ffc2bb2e, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.957160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.978082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 992/1961] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:14 2022] PUT /identity/v3/projects/08d48aaadfe34ed6a30d1f90ffbf11fa/users/2141190cb28a48198c8b4bc8ffc2bb2e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:14.989604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.989893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.990165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.029859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] MFA Rules not processed for user `2141190cb28a48198c8b4bc8ffc2bb2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:15.031439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:15.147175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 970/1962] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:15.162245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.169827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:15.236459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a2f58-fea8-4338-903f-31572041ce4c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.240390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d7140-6d01-4eed-8371-23dcdc95bd4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.241088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d7140-6d01-4eed-8371-23dcdc95bd4c service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.241618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d7140-6d01-4eed-8371-23dcdc95bd4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.242009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d7140-6d01-4eed-8371-23dcdc95bd4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.251141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d7140-6d01-4eed-8371-23dcdc95bd4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.252844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d7140-6d01-4eed-8371-23dcdc95bd4c service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.270109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 993/1963] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:15.311270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.312815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:15.327223 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:15.392981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991abed-c73f-4c63-adac-c39a4a027742 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.398687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.399411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.399781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.400119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] PATH_INFO: `/v3/users/b5610cac368348e396395ccd39ae4eee` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.401432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5610cac368348e396395ccd39ae4eee)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.403689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.472171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5610cac368348e396395ccd39ae4eee'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:15.474157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5610cac368348e396395ccd39ae4eee'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:15.482919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5610cac368348e396395ccd39ae4eee'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:15.500545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5610cac368348e396395ccd39ae4eee'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:15.501102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5610cac368348e396395ccd39ae4eee'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:15.517971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 971/1964] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:15 2022] DELETE /identity/v3/users/b5610cac368348e396395ccd39ae4eee => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:15.522725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.523035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.523312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.554148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:15.555653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:15.623397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 994/1965] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:15.629551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.632013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:15.692062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ec35-7f6e-49e9-af03-02eec01e8f17 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.696263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.696871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.697152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.697421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.706730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.707970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.723918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 972/1966] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:16.129824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.131916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:16.144742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d346d7-b2b0-4875-a0f6-8cae341a87e6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.150660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.151335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.151665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.151935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] PATH_INFO: `/v3/projects/1a0603edf6f94eb788dbd8567826cf9a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.153240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0603edf6f94eb788dbd8567826cf9a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.155154 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.211355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.213303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:16.216307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.230661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0603edf6f94eb788dbd8567826cf9a', 'request_id': 'req-1ddb55cb-b7f3-41db-8511-cab5364c063b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.234403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2744d-b420-47e9-b941-fd1ef2407b88 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.239606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0603edf6f94eb788dbd8567826cf9a', 'request_id': 'req-1ddb55cb-b7f3-41db-8511-cab5364c063b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.248011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0603edf6f94eb788dbd8567826cf9a', 'request_id': 'req-1ddb55cb-b7f3-41db-8511-cab5364c063b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.267476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] The token cache is being invalidate because project 1a0603edf6f94eb788dbd8567826cf9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:16.267995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.268708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.271335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 995/1967] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:16 2022] DELETE /identity/v3/projects/1a0603edf6f94eb788dbd8567826cf9a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:16.275690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.277097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:16.292146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.371819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.371819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.371819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.371819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] PATH_INFO: `/v3/users/994a6dcaa4614c669605721bdbf3ce61` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.372373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994a6dcaa4614c669605721bdbf3ce61)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.372373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.390587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb55cb-b7f3-41db-8511-cab5364c063b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.397942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.398998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.399286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.399736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] PATH_INFO: `/v3/users/076a1ca4d8ff40b99030f3046acbdfbc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.401068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=076a1ca4d8ff40b99030f3046acbdfbc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.404810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.448098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994a6dcaa4614c669605721bdbf3ce61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.456227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994a6dcaa4614c669605721bdbf3ce61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.469027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994a6dcaa4614c669605721bdbf3ce61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.476769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '076a1ca4d8ff40b99030f3046acbdfbc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.498182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994a6dcaa4614c669605721bdbf3ce61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.498982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9afd3ab7-4238-4e29-a945-b6763b2bb2a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994a6dcaa4614c669605721bdbf3ce61'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.517395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '076a1ca4d8ff40b99030f3046acbdfbc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.525541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 973/1968] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:16 2022] DELETE /identity/v3/users/994a6dcaa4614c669605721bdbf3ce61 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:16.529868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.530171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.530429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.531719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '076a1ca4d8ff40b99030f3046acbdfbc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.533304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '076a1ca4d8ff40b99030f3046acbdfbc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.534103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '076a1ca4d8ff40b99030f3046acbdfbc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:16.551809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 996/1969] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:16 2022] DELETE /identity/v3/users/076a1ca4d8ff40b99030f3046acbdfbc => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:16.552409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:16.553728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:16.655470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 974/1970] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:16.675427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.676785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:16.836000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.859934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33721540-aee7-427e-bd23-c876e4734768 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.867416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbdc0d-9e1a-4cb8-9fa2-d5c77661f329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.867416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbdc0d-9e1a-4cb8-9fa2-d5c77661f329 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.867858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbdc0d-9e1a-4cb8-9fa2-d5c77661f329 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.868235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbdc0d-9e1a-4cb8-9fa2-d5c77661f329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.882313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dbdc0d-9e1a-4cb8-9fa2-d5c77661f329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.884168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dbdc0d-9e1a-4cb8-9fa2-d5c77661f329 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.897379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 997/1971] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:17.010841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.012645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:17.027984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d29cc9-ee51-4afa-b83f-4700ea300ced None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.035222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.036216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.037380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.037972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] PATH_INFO: `/v3/projects/68933080acc94324bf99bf775fc7bdb9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.041707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68933080acc94324bf99bf775fc7bdb9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.045529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.140846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:17.152674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68933080acc94324bf99bf775fc7bdb9', 'request_id': 'req-d63829f8-3190-4d1f-934e-fe5663e18635'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.161454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68933080acc94324bf99bf775fc7bdb9', 'request_id': 'req-d63829f8-3190-4d1f-934e-fe5663e18635'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.176974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68933080acc94324bf99bf775fc7bdb9', 'request_id': 'req-d63829f8-3190-4d1f-934e-fe5663e18635'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.194122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] The token cache is being invalidate because project 68933080acc94324bf99bf775fc7bdb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:17.194610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.195555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.202278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 975/1972] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:17 2022] DELETE /identity/v3/projects/68933080acc94324bf99bf775fc7bdb9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:17.204094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082d9c5-05a2-4caa-b511-ce966bee54bc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.204600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082d9c5-05a2-4caa-b511-ce966bee54bc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.204695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082d9c5-05a2-4caa-b511-ce966bee54bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.232184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3082d9c5-05a2-4caa-b511-ce966bee54bc None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:17.233467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3082d9c5-05a2-4caa-b511-ce966bee54bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:17.316928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.320752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:17.370430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 998/1973] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:17.373564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e6256-c173-482c-8c10-1a2759a01822 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.373564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e6256-c173-482c-8c10-1a2759a01822 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.373564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e6256-c173-482c-8c10-1a2759a01822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.443452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dd8e6256-c173-482c-8c10-1a2759a01822 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:17.444783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd8e6256-c173-482c-8c10-1a2759a01822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:17.455532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 999/1974] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:17.461801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e6256-c173-482c-8c10-1a2759a01822 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.474277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e6256-c173-482c-8c10-1a2759a01822 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.490768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.495101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.495391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.500102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.501281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:17.501606 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.503290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.537853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1000/1975] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:17.544184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.554660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63829f8-3190-4d1f-934e-fe5663e18635 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.562311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.563535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.563939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.564279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] PATH_INFO: `/v3/projects/20eaa3ce22f6400da9f1cb5338e7422e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.565600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20eaa3ce22f6400da9f1cb5338e7422e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.567715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.582601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fb7b2-e486-4c2c-81cd-406362f9bd74 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.593292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.594424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.594718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.595662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.596744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.598293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.661904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1001/1976] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:17 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:17.665275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.682526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20eaa3ce22f6400da9f1cb5338e7422e', 'request_id': 'req-37275b24-9c2c-49ee-b271-234290632e58'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.699633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20eaa3ce22f6400da9f1cb5338e7422e', 'request_id': 'req-37275b24-9c2c-49ee-b271-234290632e58'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.722690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20eaa3ce22f6400da9f1cb5338e7422e', 'request_id': 'req-37275b24-9c2c-49ee-b271-234290632e58'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.742049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] The token cache is being invalidate because project 20eaa3ce22f6400da9f1cb5338e7422e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:17.742497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.743517 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:17.745677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 976/1977] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:17 2022] DELETE /identity/v3/projects/20eaa3ce22f6400da9f1cb5338e7422e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:17.753856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.757709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:17.818277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89ec34-32d9-408e-9a27-71aaedb7f848 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.825364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.826779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.826779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.827016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.828386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.831279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.868605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1002/1978] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:17 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:17.874930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.878703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:17.915105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab5f9e-90b6-40b4-94fc-73a1e4ea19d4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.923365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275b24-9c2c-49ee-b271-234290632e58 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.926341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.929459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.929459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.931149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.932202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.934339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.941639 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.942380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.942568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.942873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] PATH_INFO: `/v3/users/502eb8757d7a4b72b3882c2fd6b20822` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.945809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=502eb8757d7a4b72b3882c2fd6b20822)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.947998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.951018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1003/1979] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:17.955013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.962348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:17.977045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4be8d-ba72-42c7-88cd-673cc25c2ab4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.983753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.984559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.984902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.985344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.986268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.988081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.025613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1004/1980] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:18.030604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.061090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab8a04-1c70-490f-897c-0069385ef8ea admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.075155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f068018-a320-4d1a-85f1-ea304b858b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.075155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f068018-a320-4d1a-85f1-ea304b858b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.075155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f068018-a320-4d1a-85f1-ea304b858b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.075155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f068018-a320-4d1a-85f1-ea304b858b36 admin admin] PATH_INFO: `/v3/projects/5805703e2aa94450af190a5e8b3c1c3b/users/59af39407da34a49869f664e95390fab/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.075595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502eb8757d7a4b72b3882c2fd6b20822'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:18.076031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502eb8757d7a4b72b3882c2fd6b20822'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:18.092584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:18.093858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502eb8757d7a4b72b3882c2fd6b20822'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:18.107791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f068018-a320-4d1a-85f1-ea304b858b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5805703e2aa94450af190a5e8b3c1c3b, user_id=59af39407da34a49869f664e95390fab, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.110652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f068018-a320-4d1a-85f1-ea304b858b36 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.119658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502eb8757d7a4b72b3882c2fd6b20822'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:18.121653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502eb8757d7a4b72b3882c2fd6b20822'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:18.155880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1005/1981] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:18 2022] PUT /identity/v3/projects/5805703e2aa94450af190a5e8b3c1c3b/users/59af39407da34a49869f664e95390fab/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:18.157455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392c03-d5ca-488a-802e-178f925fb15e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.157834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392c03-d5ca-488a-802e-178f925fb15e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.158115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392c03-d5ca-488a-802e-178f925fb15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.176967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 977/1982] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:17 2022] DELETE /identity/v3/users/502eb8757d7a4b72b3882c2fd6b20822 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:18.205175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-47392c03-d5ca-488a-802e-178f925fb15e None None] MFA Rules not processed for user `59af39407da34a49869f664e95390fab`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:18.207238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-47392c03-d5ca-488a-802e-178f925fb15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:18.285549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1006/1983] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:18.290132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.291680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:18.366469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeb552-9611-4d2f-aea5-9d768778f2ac admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.371412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.372171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.372610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.372925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.373634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.375996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.404555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 978/1984] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:18 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:18.409017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47392c03-d5ca-488a-802e-178f925fb15e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.420994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47392c03-d5ca-488a-802e-178f925fb15e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.499718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.500687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.500976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.501242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.501910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.504218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.538945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1007/1985] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:18 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:18.543032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.557116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1ce3-9abd-436b-8c06-16c32b987baf admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.561958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.562575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.562860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.563268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.563957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.565682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.580817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 979/1986] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:18.584928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.596075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510aaf-2e3b-452c-b61d-a4bc3b73596a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.602903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.603352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.603743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.604106 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] PATH_INFO: `/v3/projects/8f36d680c3664eb6bbf1cecdf0c9d47d/users/24a46e658a33400a89a2592a1013913f/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.622555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f36d680c3664eb6bbf1cecdf0c9d47d, user_id=24a46e658a33400a89a2592a1013913f, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.625397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.649568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1008/1987] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:18 2022] PUT /identity/v3/projects/8f36d680c3664eb6bbf1cecdf0c9d47d/users/24a46e658a33400a89a2592a1013913f/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:18.653368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.668984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd230-47ea-4d85-938e-8346903a79da admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.769720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:18.779341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d440e9-7dcf-47fa-a5e3-0c9aec10adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.780369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d440e9-7dcf-47fa-a5e3-0c9aec10adce admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.780887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d440e9-7dcf-47fa-a5e3-0c9aec10adce admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.781608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d440e9-7dcf-47fa-a5e3-0c9aec10adce admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.782523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d440e9-7dcf-47fa-a5e3-0c9aec10adce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.785065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d440e9-7dcf-47fa-a5e3-0c9aec10adce admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.801860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 980/1988] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:18.806366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.833011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35677a-a888-4805-bbce-9ceed0740852 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.839334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.840139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.840476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.841298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] PATH_INFO: `/v3/projects/8f36d680c3664eb6bbf1cecdf0c9d47d/users/24a46e658a33400a89a2592a1013913f/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.843920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f36d680c3664eb6bbf1cecdf0c9d47d, user_id=24a46e658a33400a89a2592a1013913f, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.847196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.878040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1009/1989] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:18 2022] PUT /identity/v3/projects/8f36d680c3664eb6bbf1cecdf0c9d47d/users/24a46e658a33400a89a2592a1013913f/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:18.889686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.890001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.890328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.959516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] MFA Rules not processed for user `24a46e658a33400a89a2592a1013913f`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:18.963518 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.050595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.053656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.128243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:19.129084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 981/1990] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:19.141991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.143453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.194475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e8d6f-a1c4-4a48-a304-4873d6201923 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.201749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.202656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.203656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.204033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] PATH_INFO: `/v3/projects/17a3c1805a4a493dbf65d4fe0df43ea3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.205309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a3c1805a4a493dbf65d4fe0df43ea3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.207394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.225661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104a972-b86a-4dcb-9955-893f71e5ed35 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.229965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.230834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.231259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.231760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.244064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.246232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.257653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 982/1991] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:19.258818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.260596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.264079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a3c1805a4a493dbf65d4fe0df43ea3', 'request_id': 'req-c14d3f80-b40f-4efa-a970-e12973664012'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.272062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a3c1805a4a493dbf65d4fe0df43ea3', 'request_id': 'req-c14d3f80-b40f-4efa-a970-e12973664012'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.279026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f293a75-612e-457d-a760-749924b17f3d service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.281645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a3c1805a4a493dbf65d4fe0df43ea3', 'request_id': 'req-c14d3f80-b40f-4efa-a970-e12973664012'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.300082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] The token cache is being invalidate because project 17a3c1805a4a493dbf65d4fe0df43ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:19.300596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.301831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c14d3f80-b40f-4efa-a970-e12973664012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.304413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1010/1992] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:19 2022] DELETE /identity/v3/projects/17a3c1805a4a493dbf65d4fe0df43ea3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:19.309911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0adb2-fbe5-407f-9b16-fb35c44bd75f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.311241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0adb2-fbe5-407f-9b16-fb35c44bd75f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.311692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0adb2-fbe5-407f-9b16-fb35c44bd75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.353998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bce0adb2-fbe5-407f-9b16-fb35c44bd75f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:19.355250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bce0adb2-fbe5-407f-9b16-fb35c44bd75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.386179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.386978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.387262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.387695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] PATH_INFO: `/v3/users/2141190cb28a48198c8b4bc8ffc2bb2e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.389044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2141190cb28a48198c8b4bc8ffc2bb2e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.391051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.421744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:19.482197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1011/1993] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:19.483988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2141190cb28a48198c8b4bc8ffc2bb2e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.484848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2141190cb28a48198c8b4bc8ffc2bb2e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.497364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca269944-958a-45de-9c32-7813c7f3ff40 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.497364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca269944-958a-45de-9c32-7813c7f3ff40 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.497364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca269944-958a-45de-9c32-7813c7f3ff40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.499456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2141190cb28a48198c8b4bc8ffc2bb2e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.516776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2141190cb28a48198c8b4bc8ffc2bb2e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.517487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bdbde1b2-dfa6-489c-8202-5e77f4aa8155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2141190cb28a48198c8b4bc8ffc2bb2e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:19.544479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ca269944-958a-45de-9c32-7813c7f3ff40 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:19.545395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ca269944-958a-45de-9c32-7813c7f3ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.556422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1012/1994] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:19.566891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca269944-958a-45de-9c32-7813c7f3ff40 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.570253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 983/1995] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:19 2022] DELETE /identity/v3/users/2141190cb28a48198c8b4bc8ffc2bb2e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:19.575107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.575911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.576344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.581466 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca269944-958a-45de-9c32-7813c7f3ff40 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.632372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:19.634403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.660904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:19.691457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.692385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.694182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.694621 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.695339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:19.697854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.705626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.750452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1013/1996] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:19.757692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.770544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 984/1997] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:19.783414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c098f-a8cc-4f50-912d-40ee9d3107a6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.786047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.788645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:19.794182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.796211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.796492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.796763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.797672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.809213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.851623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1014/1998] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:19 2022] POST /identity/v3/projects => generated 361 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:19.854993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.917713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874a3f6-8114-4cbc-9709-f77536962b00 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.954712 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fb62a-6763-48d6-b074-7ae6e8912809 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.960523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.961397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.961834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.962263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.963080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.965227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.981465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.982167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.982553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.982860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.001352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1015/1999] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:19 2022] POST /identity/v3/users => generated 281 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:20.001352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.002909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.006234 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.006424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.017293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:20.027862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad657df-b4b1-4af1-927a-5355096445b6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.031627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 985/2000] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:20.033592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.036806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.037654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.037745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.038996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] PATH_INFO: `/v3/users/59af39407da34a49869f664e95390fab` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.040845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=59af39407da34a49869f664e95390fab)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.044500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.054931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbde124-b4ff-4d10-b965-7e4d893cf91c service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.061541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.062353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.062707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.063054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.063815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.065508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.082027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 986/2001] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:20.082867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.094548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe4339-758c-495d-a889-5c6a7f04e906 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.103796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.105235 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.105750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.106303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.107319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.109956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.126389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 987/2002] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:20.127809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.134797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59af39407da34a49869f664e95390fab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.144321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6820133-2a39-4337-a2fd-d0da0e981b41 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.153740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f0a6-2223-4fa4-a528-c3815ebe3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.153956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59af39407da34a49869f664e95390fab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.154648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f0a6-2223-4fa4-a528-c3815ebe3460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.155278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f0a6-2223-4fa4-a528-c3815ebe3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.155667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f0a6-2223-4fa4-a528-c3815ebe3460 admin admin] PATH_INFO: `/v3/projects/8194b447dde648949a5bb46294838b83/users/6fdbf089aee94b52b863f4a5799af3b4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.185607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59af39407da34a49869f664e95390fab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.186716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59af39407da34a49869f664e95390fab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.187707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9459d92a-f018-4b27-b24b-d9222801d3ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59af39407da34a49869f664e95390fab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.190976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f0a6-2223-4fa4-a528-c3815ebe3460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8194b447dde648949a5bb46294838b83, user_id=6fdbf089aee94b52b863f4a5799af3b4, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.196057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f0a6-2223-4fa4-a528-c3815ebe3460 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.214026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1016/2003] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:19 2022] DELETE /identity/v3/users/59af39407da34a49869f664e95390fab => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:20.216754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 988/2004] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:20 2022] PUT /identity/v3/projects/8194b447dde648949a5bb46294838b83/users/6fdbf089aee94b52b863f4a5799af3b4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:20.218598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.218891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.219162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.230102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.230635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.231242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.257047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:20.258303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.263922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] MFA Rules not processed for user `6fdbf089aee94b52b863f4a5799af3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:20.265655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.294960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:20.363758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1017/2005] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:20.379195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 989/2006] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:20.388976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.390767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.540172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b7c03-2372-4f09-95f4-71b4e5207d43 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.585093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.588997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.589291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.589573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.641362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.641873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.647470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.656419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.679627 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b66b1-f550-4205-8ef2-ead29c05e050 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.697480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.698193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.698424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.698704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] PATH_INFO: `/v3/projects/08d48aaadfe34ed6a30d1f90ffbf11fa` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.704815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1018/2007] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:20.705445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d48aaadfe34ed6a30d1f90ffbf11fa)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.710928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.737897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.739427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.836418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d48aaadfe34ed6a30d1f90ffbf11fa', 'request_id': 'req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.839291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41445f44-2167-44d1-8f75-cc31bfd9712a service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.848139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d48aaadfe34ed6a30d1f90ffbf11fa', 'request_id': 'req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.863730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d48aaadfe34ed6a30d1f90ffbf11fa', 'request_id': 'req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.886889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] The token cache is being invalidate because project 08d48aaadfe34ed6a30d1f90ffbf11fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:20.888669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.890182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae3987b4-69e4-4ceb-b5a8-ac7aafe043cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:20.892785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 990/2008] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:20 2022] DELETE /identity/v3/projects/08d48aaadfe34ed6a30d1f90ffbf11fa => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:20.897401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf822e-c21c-4d8d-b8a0-95a0b1a24158 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.897705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf822e-c21c-4d8d-b8a0-95a0b1a24158 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.898038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf822e-c21c-4d8d-b8a0-95a0b1a24158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.907648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.909306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.909844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.910320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.912930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.921650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0daf822e-c21c-4d8d-b8a0-95a0b1a24158 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:20.922816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0daf822e-c21c-4d8d-b8a0-95a0b1a24158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.958836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.001900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.044592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.047338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.051634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 991/2009] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:21.058993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ee73-71d6-423d-9bce-27c344ebd27b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.059767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ee73-71d6-423d-9bce-27c344ebd27b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.060065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ee73-71d6-423d-9bce-27c344ebd27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.079339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1019/2010] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:21.088053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.089824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.103698 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-cb93ee73-71d6-423d-9bce-27c344ebd27b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:21.105184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cb93ee73-71d6-423d-9bce-27c344ebd27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.117478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 992/2011] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:21.118914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6307cb-9a71-4e94-b7b6-0d3991420b47 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.124595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93ee73-71d6-423d-9bce-27c344ebd27b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.132162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.133133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.133522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.133928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] PATH_INFO: `/v3/users/6fdbf089aee94b52b863f4a5799af3b4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.136112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fdbf089aee94b52b863f4a5799af3b4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.138177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.147876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93ee73-71d6-423d-9bce-27c344ebd27b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.162005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.162711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.163703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.164321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.165044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:21.166525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.168902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.201173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 993/2012] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:21.201956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.221604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2f324-4341-43d5-8691-9ba7247848e7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.239550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.242050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.244823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.245376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.246579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.248825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.250877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fdbf089aee94b52b863f4a5799af3b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.264156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fdbf089aee94b52b863f4a5799af3b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.288265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fdbf089aee94b52b863f4a5799af3b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.289423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fdbf089aee94b52b863f4a5799af3b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.290228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fdbf089aee94b52b863f4a5799af3b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.310196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 994/2013] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:21 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:21.311199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.312948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.314869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1020/2014] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:21 2022] DELETE /identity/v3/users/6fdbf089aee94b52b863f4a5799af3b4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:21.315864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.317352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.423060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b0c97-6e0b-4772-a403-6cd92f585879 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.435868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59240ad-af82-4f14-9a2c-3f01649e6c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.436500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59240ad-af82-4f14-9a2c-3f01649e6c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.436779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59240ad-af82-4f14-9a2c-3f01649e6c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.437046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59240ad-af82-4f14-9a2c-3f01649e6c41 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.437692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59240ad-af82-4f14-9a2c-3f01649e6c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.439722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59240ad-af82-4f14-9a2c-3f01649e6c41 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.440609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e84639-eef8-428d-a608-b797abf34639 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.446890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.447819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.448250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.448685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] PATH_INFO: `/v3/projects/5805703e2aa94450af190a5e8b3c1c3b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.449897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5805703e2aa94450af190a5e8b3c1c3b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.452049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.467979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1021/2015] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:21 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:21.470817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe2cb4-2af8-48c9-acb8-01b034ed5bd2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.471269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe2cb4-2af8-48c9-acb8-01b034ed5bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.471832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe2cb4-2af8-48c9-acb8-01b034ed5bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.495889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7abe2cb4-2af8-48c9-acb8-01b034ed5bd2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:21.497514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7abe2cb4-2af8-48c9-acb8-01b034ed5bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.508004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1022/2016] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:21.512547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe2cb4-2af8-48c9-acb8-01b034ed5bd2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.526001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe2cb4-2af8-48c9-acb8-01b034ed5bd2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.549647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5805703e2aa94450af190a5e8b3c1c3b', 'request_id': 'req-b2dbbc38-f93c-4352-8e56-09cd5eace0be'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.566340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5805703e2aa94450af190a5e8b3c1c3b', 'request_id': 'req-b2dbbc38-f93c-4352-8e56-09cd5eace0be'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.570111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.579726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5805703e2aa94450af190a5e8b3c1c3b', 'request_id': 'req-b2dbbc38-f93c-4352-8e56-09cd5eace0be'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.598045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] The token cache is being invalidate because project 5805703e2aa94450af190a5e8b3c1c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:21.598640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.600823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.604386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 995/2017] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:21 2022] DELETE /identity/v3/projects/5805703e2aa94450af190a5e8b3c1c3b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:21.605202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.607438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.626064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.626701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.626983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.627256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.627944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.629565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.643855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1023/2018] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:21.647535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.649034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.650056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.667817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61abdd-6ec1-46ac-8b3c-0a23085df3c9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.676803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.678129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.678406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.678675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] PATH_INFO: `/v3/users/24a46e658a33400a89a2592a1013913f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.680508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a46e658a33400a89a2592a1013913f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.683567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.745017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbbc38-f93c-4352-8e56-09cd5eace0be admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.752754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.753486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.753768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.754083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.756216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.771913 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.773360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.786942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 996/2019] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:21.799051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.804962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.824688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a46e658a33400a89a2592a1013913f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.830699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0ed7f-eecc-44c6-a278-801e3bbb0fbd service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.838188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.838822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.839239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.840115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.841516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.843931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.847236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a46e658a33400a89a2592a1013913f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.865405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 997/2020] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:21.869793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.883964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcbbeb-7274-4206-9e29-987ea12bce68 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.887169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a46e658a33400a89a2592a1013913f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.889410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.889982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a46e658a33400a89a2592a1013913f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.890187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.890472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.890746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] PATH_INFO: `/v3/projects/70d39d21d2844c3599629db4abdc1600/users/cf0dc39479d1435f902e10cb6827f5e4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.892851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c090340-288d-4cfa-b75c-4ffcbfa93ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a46e658a33400a89a2592a1013913f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:21.921013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70d39d21d2844c3599629db4abdc1600, user_id=cf0dc39479d1435f902e10cb6827f5e4, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.924663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.931981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1024/2021] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:21 2022] DELETE /identity/v3/users/24a46e658a33400a89a2592a1013913f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:21.952577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 998/2022] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:21 2022] PUT /identity/v3/projects/70d39d21d2844c3599629db4abdc1600/users/cf0dc39479d1435f902e10cb6827f5e4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:21.953393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.953776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.954150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.990687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:21.992169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.129090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1025/2023] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:22.172076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.174941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.277772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1838e2-4aeb-462a-807d-6be3cede8cc7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.287133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1f17f-2cee-4090-ac62-be49f1b09610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.287804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1f17f-2cee-4090-ac62-be49f1b09610 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.288086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1f17f-2cee-4090-ac62-be49f1b09610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.288365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1f17f-2cee-4090-ac62-be49f1b09610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.343980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1f17f-2cee-4090-ac62-be49f1b09610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.347674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1f17f-2cee-4090-ac62-be49f1b09610 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.377479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 999/2024] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:22.484420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.486078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.516973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd366a39-c218-49b1-9ce0-06ba81240c01 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.529006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.529723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.530004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.530281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] PATH_INFO: `/v3/projects/8194b447dde648949a5bb46294838b83` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.532592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8194b447dde648949a5bb46294838b83)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.534959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.588388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.600297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8194b447dde648949a5bb46294838b83', 'request_id': 'req-a26346d8-42d3-4fef-b583-8b126796af57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:22.622643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8194b447dde648949a5bb46294838b83', 'request_id': 'req-a26346d8-42d3-4fef-b583-8b126796af57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:22.635241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8194b447dde648949a5bb46294838b83', 'request_id': 'req-a26346d8-42d3-4fef-b583-8b126796af57'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:22.652054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] The token cache is being invalidate because project 8194b447dde648949a5bb46294838b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:22.652429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:22.653232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a26346d8-42d3-4fef-b583-8b126796af57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:22.655125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1026/2025] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:22 2022] DELETE /identity/v3/projects/8194b447dde648949a5bb46294838b83 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:22.659908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f1cd7-5952-4e5c-894f-cf4dd869d0fb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.660389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f1cd7-5952-4e5c-894f-cf4dd869d0fb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.660478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f1cd7-5952-4e5c-894f-cf4dd869d0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.683784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7d8f1cd7-5952-4e5c-894f-cf4dd869d0fb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:22.685019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7d8f1cd7-5952-4e5c-894f-cf4dd869d0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.827007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.827200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1000/2026] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:22.827787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.828240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.875723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:22.877585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.892867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1027/2027] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:22.897822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f1cd7-5952-4e5c-894f-cf4dd869d0fb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.912187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f1cd7-5952-4e5c-894f-cf4dd869d0fb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.918792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.919665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.919956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.920280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.922094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:22.925352 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.925884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.928354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.930204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.949534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1001/2028] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:22.950563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.954554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ec28-23b5-4094-b502-8f5c4248baf4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.963122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0c754-4783-4147-94f4-16195740813e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.963730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.964752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.965873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.966362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] PATH_INFO: `/v3/projects/8f36d680c3664eb6bbf1cecdf0c9d47d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.968134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f36d680c3664eb6bbf1cecdf0c9d47d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.968283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.968985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.969270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.969546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.970520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.971798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.972499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.988671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.017522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1002/2029] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:22 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.026211 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.053195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f36d680c3664eb6bbf1cecdf0c9d47d', 'request_id': 'req-16b1e0e3-61c9-4e83-925c-bf61843c2903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:23.069806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f36d680c3664eb6bbf1cecdf0c9d47d', 'request_id': 'req-16b1e0e3-61c9-4e83-925c-bf61843c2903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:23.094050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f36d680c3664eb6bbf1cecdf0c9d47d', 'request_id': 'req-16b1e0e3-61c9-4e83-925c-bf61843c2903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:23.116689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] The token cache is being invalidate because project 8f36d680c3664eb6bbf1cecdf0c9d47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:23.117962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:23.120572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-16b1e0e3-61c9-4e83-925c-bf61843c2903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:23.129153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1028/2030] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:22 2022] DELETE /identity/v3/projects/8f36d680c3664eb6bbf1cecdf0c9d47d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:23.135706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22138f8b-9e2c-4889-b614-39dd20ad09be None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.136028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22138f8b-9e2c-4889-b614-39dd20ad09be None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.136836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22138f8b-9e2c-4889-b614-39dd20ad09be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.142581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13597d11-194e-455b-8944-341a1901e2b2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.175524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-22138f8b-9e2c-4889-b614-39dd20ad09be None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:23.177414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-22138f8b-9e2c-4889-b614-39dd20ad09be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:23.221417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.222345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.222770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.223291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.224810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.230389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.261152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1003/2031] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.264934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.268825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:23.286206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1029/2032] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:23.289440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b5aa4-94a1-4613-a9ed-45264e6ffd6f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.289741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b5aa4-94a1-4613-a9ed-45264e6ffd6f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.290297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b5aa4-94a1-4613-a9ed-45264e6ffd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.294642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d73100-fe54-47c8-b4e9-e71561e62f69 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.303180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.303870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.304156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.304429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.305082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.306864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.321039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-df5b5aa4-94a1-4613-a9ed-45264e6ffd6f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:23.322347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-df5b5aa4-94a1-4613-a9ed-45264e6ffd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:23.324490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1004/2033] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:23.325547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.334868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1030/2034] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:23.335916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b5aa4-94a1-4613-a9ed-45264e6ffd6f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.340049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923001a-3dac-41a2-bbb1-0a232725f965 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.346869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.347064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b5aa4-94a1-4613-a9ed-45264e6ffd6f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.347575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.347801 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.348078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.348727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.351053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.353430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.354522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.354911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.355386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.356261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:23.356749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.361358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.364607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1005/2035] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:23.369448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.379730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1031/2036] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:23.381352 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7a996-f0b4-4f3b-9015-333e2ceb01d4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.384239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.387342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7d69f-1e84-482f-9bee-d7914ff2da3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.388239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d69f-1e84-482f-9bee-d7914ff2da3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.388895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d69f-1e84-482f-9bee-d7914ff2da3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.389009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d69f-1e84-482f-9bee-d7914ff2da3a admin admin] PATH_INFO: `/v3/projects/2c46f2aef6d34466952e40343ab9ad17/users/9b2bf733b0264aa29c6ddc37acee1e0e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.398248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436c870-477b-4ee3-bdb5-34bd8aca887e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.403365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.404006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.404283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.404553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.405184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.406336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7d69f-1e84-482f-9bee-d7914ff2da3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c46f2aef6d34466952e40343ab9ad17, user_id=9b2bf733b0264aa29c6ddc37acee1e0e, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.407208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.409972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7d69f-1e84-482f-9bee-d7914ff2da3a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.437967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1006/2037] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:23 2022] PUT /identity/v3/projects/2c46f2aef6d34466952e40343ab9ad17/users/9b2bf733b0264aa29c6ddc37acee1e0e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:23.440510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfead5-0c40-4b83-a660-558948a3e2ed None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.441037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfead5-0c40-4b83-a660-558948a3e2ed None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.441411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfead5-0c40-4b83-a660-558948a3e2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.446191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1032/2038] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.450397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.456126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.491341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-eebfead5-0c40-4b83-a660-558948a3e2ed None None] MFA Rules not processed for user `9b2bf733b0264aa29c6ddc37acee1e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:23.492638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eebfead5-0c40-4b83-a660-558948a3e2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:23.525188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.547841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111914d-82ac-443b-8ee8-17f1d72117c5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.554372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.555106 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.555568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.555836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.556472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.558532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.599145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1007/2039] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:23.609558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1033/2040] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.613830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.626017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfead5-0c40-4b83-a660-558948a3e2ed None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.629242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13f62d-ed7b-4eb5-a998-1d6f476e8326 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.638215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.638844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.639640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.639968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.640629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.643553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.650811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfead5-0c40-4b83-a660-558948a3e2ed None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.662274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1034/2041] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:23.667084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.669156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.670466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.671136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.671136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.671780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.673884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.687259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db048360-5477-40c1-8df6-5f55bee348a5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.692273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.693652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.694041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.694398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.695143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.696909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.711480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1035/2042] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:23.721800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.741975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb756-f078-4f89-937c-11a94ba0c917 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.744083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1008/2043] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.745372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.830742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041354e8-69a0-44fe-8c81-69c385e3281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.830983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041354e8-69a0-44fe-8c81-69c385e3281d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.831376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041354e8-69a0-44fe-8c81-69c385e3281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.831683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041354e8-69a0-44fe-8c81-69c385e3281d admin admin] PATH_INFO: `/v3/projects/1f3f4603d9ae469daac3ab8e21cdcd9d/users/da6f663cab13436a8ddc6e80d37c125e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.845053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338120d4-3d99-4794-9bee-1f07765bd68b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.854919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041354e8-69a0-44fe-8c81-69c385e3281d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f3f4603d9ae469daac3ab8e21cdcd9d, user_id=da6f663cab13436a8ddc6e80d37c125e, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.856338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.857017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.857297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.857568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.858194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041354e8-69a0-44fe-8c81-69c385e3281d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.858312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.861303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.885209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1036/2044] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:23 2022] PUT /identity/v3/projects/1f3f4603d9ae469daac3ab8e21cdcd9d/users/da6f663cab13436a8ddc6e80d37c125e/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:23.894741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6aa40f-2b5c-4cf4-a8f8-9cc82dcde7f1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.895222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6aa40f-2b5c-4cf4-a8f8-9cc82dcde7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.895596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6aa40f-2b5c-4cf4-a8f8-9cc82dcde7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.915750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1009/2045] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.920064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.956629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9011a-13c0-4e3c-9652-171960b1680d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.984941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7e6aa40f-2b5c-4cf4-a8f8-9cc82dcde7f1 None None] MFA Rules not processed for user `da6f663cab13436a8ddc6e80d37c125e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:23.984941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e6aa40f-2b5c-4cf4-a8f8-9cc82dcde7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:24.078314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.078978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.079262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.079649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.080350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.081935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.082247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.108405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1010/2046] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:24.113657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.152405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6fd8-8c95-4a3f-aaf6-cc3d4f6698df admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.164022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1037/2047] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:24.165243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6aa40f-2b5c-4cf4-a8f8-9cc82dcde7f1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.169358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.172336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.175304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.175623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.178238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.180068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.189678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6aa40f-2b5c-4cf4-a8f8-9cc82dcde7f1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.193476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1011/2048] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:24.196064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.197577 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.198254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.198363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.198956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.200099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.202578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.218472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7b48-b5d0-418f-b256-9ada74460973 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.227726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867d3f3-8cc5-4d5c-8842-3603ebd190ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.229276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867d3f3-8cc5-4d5c-8842-3603ebd190ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.230546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867d3f3-8cc5-4d5c-8842-3603ebd190ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.231114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867d3f3-8cc5-4d5c-8842-3603ebd190ad admin admin] PATH_INFO: `/v3/projects/5f9b6d607097418cbbee932d992a1ea9/users/078e1dcc9cfe4d0595770bcfa75bf0b4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.244635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1038/2049] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:24 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:24.245803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.264079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867d3f3-8cc5-4d5c-8842-3603ebd190ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9b6d607097418cbbee932d992a1ea9, user_id=078e1dcc9cfe4d0595770bcfa75bf0b4, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.271139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867d3f3-8cc5-4d5c-8842-3603ebd190ad admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.296330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.316690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1012/2050] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:24 2022] PUT /identity/v3/projects/5f9b6d607097418cbbee932d992a1ea9/users/078e1dcc9cfe4d0595770bcfa75bf0b4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:24.329556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d7149-2613-4541-bbe6-00518e236b16 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.330772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d7149-2613-4541-bbe6-00518e236b16 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.331412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d7149-2613-4541-bbe6-00518e236b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.346542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab9c51-5e83-4f10-b428-398ab6e393d3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.430344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd30078-6457-46a8-bdf7-81435079197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.430791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9c9d7149-2613-4541-bbe6-00518e236b16 None None] MFA Rules not processed for user `078e1dcc9cfe4d0595770bcfa75bf0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:24.430970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd30078-6457-46a8-bdf7-81435079197d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.431255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd30078-6457-46a8-bdf7-81435079197d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.431557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd30078-6457-46a8-bdf7-81435079197d admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.432210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd30078-6457-46a8-bdf7-81435079197d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.433457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9c9d7149-2613-4541-bbe6-00518e236b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:24.434147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd30078-6457-46a8-bdf7-81435079197d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.462928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1039/2051] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:24 2022] POST /identity/v3/users => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:24.465105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5aa2-c786-4d8c-bf13-c7129f2f160e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.465401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5aa2-c786-4d8c-bf13-c7129f2f160e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.465747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5aa2-c786-4d8c-bf13-c7129f2f160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.505825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f8ff5aa2-c786-4d8c-bf13-c7129f2f160e None None] MFA Rules not processed for user `cf0dc39479d1435f902e10cb6827f5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:24.507716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f8ff5aa2-c786-4d8c-bf13-c7129f2f160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:24.563944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.596348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1013/2052] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:24.597325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d7149-2613-4541-bbe6-00518e236b16 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.606489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1040/2053] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:24.611180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5aa2-c786-4d8c-bf13-c7129f2f160e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.614350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d7149-2613-4541-bbe6-00518e236b16 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.622933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5aa2-c786-4d8c-bf13-c7129f2f160e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.627261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.628091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.628840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.629515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.629802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.629905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.630644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.631320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.633349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.633724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.634474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.636233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.654761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1014/2054] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:24.655917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.660029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:24.672512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1041/2055] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:24 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:24.673835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.734759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf80cc-52a7-4793-842c-bf66eb65ba84 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.792792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccfb91-8657-48b0-84cd-5de442547527 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.825982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.826867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.827212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.827573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] PATH_INFO: `/v3/projects/0952490ef6754acdba5d2346ea566ad4/users/904eeea82220460db89b4694f90fbfc5/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.835275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.835898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.836186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.836462 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.870780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0952490ef6754acdba5d2346ea566ad4, user_id=904eeea82220460db89b4694f90fbfc5, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.874485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.900011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1042/2056] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:24 2022] PUT /identity/v3/projects/0952490ef6754acdba5d2346ea566ad4/users/904eeea82220460db89b4694f90fbfc5/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:24.901330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.946993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.967632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.969255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.986235 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1015/2057] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:24.988569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.005893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966c48c-02f7-4c59-a053-d473ef3b2bc2 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.054521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b06388-b2a1-49b5-aed7-5ee3a0505a17 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.065807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a74c07-1bc6-4efa-a4be-d1d7a1455e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.066498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a74c07-1bc6-4efa-a4be-d1d7a1455e18 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.066856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a74c07-1bc6-4efa-a4be-d1d7a1455e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.067201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a74c07-1bc6-4efa-a4be-d1d7a1455e18 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.068505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a74c07-1bc6-4efa-a4be-d1d7a1455e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.070615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a74c07-1bc6-4efa-a4be-d1d7a1455e18 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.119546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1043/2058] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:24 2022] POST /identity/v3/users => generated 298 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:25.120979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a670e7-6f04-469e-8d57-4b03df8d9cb3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.122685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a670e7-6f04-469e-8d57-4b03df8d9cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.123238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a670e7-6f04-469e-8d57-4b03df8d9cb3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.124811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1044/2059] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:03:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:25.128494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a670e7-6f04-469e-8d57-4b03df8d9cb3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.133769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33d5eb-66ec-4f62-acdf-33dd06b27102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.134606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33d5eb-66ec-4f62-acdf-33dd06b27102 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.136277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33d5eb-66ec-4f62-acdf-33dd06b27102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.136704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33d5eb-66ec-4f62-acdf-33dd06b27102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.138590 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33d5eb-66ec-4f62-acdf-33dd06b27102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.148196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33d5eb-66ec-4f62-acdf-33dd06b27102 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.168456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a670e7-6f04-469e-8d57-4b03df8d9cb3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.180196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.183271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.183888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1016/2060] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:25.187836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.188172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.188289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f76d65-29db-41c0-9827-516d1b044c43 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.188480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f76d65-29db-41c0-9827-516d1b044c43 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.188739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f76d65-29db-41c0-9827-516d1b044c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.188853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.196581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.222894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1045/2061] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:25.222894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.233568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e76a2-2cdb-4319-bc28-4d90c227dc19 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.241128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.241811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.242116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.242389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] PATH_INFO: `/v3/projects/0952490ef6754acdba5d2346ea566ad4/users/904eeea82220460db89b4694f90fbfc5/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.243979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-46f76d65-29db-41c0-9827-516d1b044c43 None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:25.244476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0952490ef6754acdba5d2346ea566ad4, user_id=904eeea82220460db89b4694f90fbfc5, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.248103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-46f76d65-29db-41c0-9827-516d1b044c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:25.248411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.273170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1046/2062] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:25 2022] PUT /identity/v3/projects/0952490ef6754acdba5d2346ea566ad4/users/904eeea82220460db89b4694f90fbfc5/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:25.274265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.305547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:25.384954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1017/2063] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:03:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:25.386109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be13a1-1c70-443f-a039-d38b90a84163 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.386400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be13a1-1c70-443f-a039-d38b90a84163 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.386679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be13a1-1c70-443f-a039-d38b90a84163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.423454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-11be13a1-1c70-443f-a039-d38b90a84163 None None] MFA Rules not processed for user `904eeea82220460db89b4694f90fbfc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:25.424750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-11be13a1-1c70-443f-a039-d38b90a84163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:25.425285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31a13-2007-4997-bbc2-d91c96e1b4e2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.432738 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.434316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.435611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.436174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] PATH_INFO: `/v3/projects/8c720c14c32a4ff792b1548215df8e1c/users/5c7a2a82e86d468780fa14a61c7eb33e/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.473239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c720c14c32a4ff792b1548215df8e1c, user_id=5c7a2a82e86d468780fa14a61c7eb33e, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.476674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.500233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1047/2064] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:25 2022] PUT /identity/v3/projects/8c720c14c32a4ff792b1548215df8e1c/users/5c7a2a82e86d468780fa14a61c7eb33e/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:25.501391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.503561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:25.523687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1018/2065] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:25.525359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be13a1-1c70-443f-a039-d38b90a84163 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.540599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be13a1-1c70-443f-a039-d38b90a84163 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.546285 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:25.626534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c77f2-3e00-4a2b-9bdd-cee8946c8faf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.636073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.636658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.636942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.637220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.652148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.653797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.665867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1048/2066] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:25.667150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.668909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.669860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.670324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.670806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.671885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.674210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.691092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1019/2067] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:25.696985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.709041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89673c0-9383-4215-a530-7114335acf63 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.721727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199bdae-5300-460e-8e61-b5a503b2a6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.722533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199bdae-5300-460e-8e61-b5a503b2a6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.722822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199bdae-5300-460e-8e61-b5a503b2a6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.723100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199bdae-5300-460e-8e61-b5a503b2a6b1 admin admin] PATH_INFO: `/v3/projects/8c720c14c32a4ff792b1548215df8e1c/users/5c7a2a82e86d468780fa14a61c7eb33e/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.726204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199bdae-5300-460e-8e61-b5a503b2a6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c720c14c32a4ff792b1548215df8e1c, user_id=5c7a2a82e86d468780fa14a61c7eb33e, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.735430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199bdae-5300-460e-8e61-b5a503b2a6b1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.749269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1020/2068] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:25 2022] PUT /identity/v3/projects/8c720c14c32a4ff792b1548215df8e1c/users/5c7a2a82e86d468780fa14a61c7eb33e/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:25.759317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a027de-c030-4459-86ee-06beceada682 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.759655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a027de-c030-4459-86ee-06beceada682 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.759940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a027de-c030-4459-86ee-06beceada682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.771782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0258-f4c4-407a-9ff8-9b0b1ad28626 service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.797153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-60a027de-c030-4459-86ee-06beceada682 None None] MFA Rules not processed for user `5c7a2a82e86d468780fa14a61c7eb33e`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:25.798757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-60a027de-c030-4459-86ee-06beceada682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:25.823635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:25.840379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.841098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.841379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.842881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.905374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1021/2069] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:25.918466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a027de-c030-4459-86ee-06beceada682 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.920467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-60a027de-c030-4459-86ee-06beceada682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:25.929633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.931108 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.933743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a027de-c030-4459-86ee-06beceada682 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.937473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24947-29f8-4640-bb42-87de5b7d7808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.938200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24947-29f8-4640-bb42-87de5b7d7808 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.938573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24947-29f8-4640-bb42-87de5b7d7808 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.938939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24947-29f8-4640-bb42-87de5b7d7808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.953446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1049/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:26.014449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be24947-29f8-4640-bb42-87de5b7d7808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.017440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be24947-29f8-4640-bb42-87de5b7d7808 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.030574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1022/2071] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:26.043481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.052124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402d850-17e2-40e9-86c1-74812647a452 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.056655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.057477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.057632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.057891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.065459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768123-0e4d-4bbd-a877-29e16f88a43d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.065815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768123-0e4d-4bbd-a877-29e16f88a43d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.066170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768123-0e4d-4bbd-a877-29e16f88a43d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.068422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1023/2072] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:03:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:26.088232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a7f47-87f4-4ee3-a715-45d67fac78d3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.088775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a7f47-87f4-4ee3-a715-45d67fac78d3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.090186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a7f47-87f4-4ee3-a715-45d67fac78d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.117797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-077a7f47-87f4-4ee3-a715-45d67fac78d3 None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:26.120247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-077a7f47-87f4-4ee3-a715-45d67fac78d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.124166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.126012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.137605 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.141185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1050/2073] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:26.170223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.173005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.236723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1024/2074] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:03:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:26.253523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a7f47-87f4-4ee3-a715-45d67fac78d3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.266997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a7f47-87f4-4ee3-a715-45d67fac78d3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.276295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.276295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.276452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.276890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.278291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651c98a-dc21-42ab-81ed-20ce24a4abbf service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.286251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.287481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.288036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.288248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.289627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.289627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] PATH_INFO: `/v3/users/d5e7cd285e084f4cb1e5249489216992` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.291476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e7cd285e084f4cb1e5249489216992)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.293534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.308964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1025/2075] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:26.365922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.383256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e7cd285e084f4cb1e5249489216992'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.397999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e7cd285e084f4cb1e5249489216992'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.420447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e7cd285e084f4cb1e5249489216992'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.429635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e7cd285e084f4cb1e5249489216992'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.431131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-954bf66e-53d1-48eb-a989-71072bb7ad20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e7cd285e084f4cb1e5249489216992'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.461162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1051/2076] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:26 2022] DELETE /identity/v3/users/d5e7cd285e084f4cb1e5249489216992 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:26.597387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.609000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ca083-f870-4bd1-b57e-85bbfd422873 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.648718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.649232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.649350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.672294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] MFA Rules not processed for user `cf0dc39479d1435f902e10cb6827f5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:26.673978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.686229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.686872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.687146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.687416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] PATH_INFO: `/v3/users/da6f663cab13436a8ddc6e80d37c125e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.688608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6f663cab13436a8ddc6e80d37c125e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.690667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.747589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1052/2077] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:26.747589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.750704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6f663cab13436a8ddc6e80d37c125e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.751168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.751375 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6f663cab13436a8ddc6e80d37c125e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.760159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6f663cab13436a8ddc6e80d37c125e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.767089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db956afe-a93a-4d12-8615-bb6fd57000dc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.772150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.772784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.773067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.773335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] PATH_INFO: `/v3/projects/15cf90bead99458badb5625fababfebc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.774607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15cf90bead99458badb5625fababfebc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.775541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6f663cab13436a8ddc6e80d37c125e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.776646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.777026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b98c1a21-9b92-48f5-ac51-150a93ec98a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6f663cab13436a8ddc6e80d37c125e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.795035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1026/2078] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:26 2022] DELETE /identity/v3/users/da6f663cab13436a8ddc6e80d37c125e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:26.796375 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.796732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.797336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.818571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:26.819967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.829752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.841257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15cf90bead99458badb5625fababfebc', 'request_id': 'req-7eee11fd-8d0e-4bb8-a486-1266dc174f43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.852153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15cf90bead99458badb5625fababfebc', 'request_id': 'req-7eee11fd-8d0e-4bb8-a486-1266dc174f43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.858259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15cf90bead99458badb5625fababfebc', 'request_id': 'req-7eee11fd-8d0e-4bb8-a486-1266dc174f43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.874114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] The token cache is being invalidate because project 15cf90bead99458badb5625fababfebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:26.874575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.875605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:26.877507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1053/2079] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:26 2022] DELETE /identity/v3/projects/15cf90bead99458badb5625fababfebc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:26.881238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.882702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.887398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1027/2080] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:26.899586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.901481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.929480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.956836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee11fd-8d0e-4bb8-a486-1266dc174f43 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.962229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.962853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.963126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.963581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] PATH_INFO: `/v3/users/46436bf8ece14df0a01d637fecc57543` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.964789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46436bf8ece14df0a01d637fecc57543)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.967149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.969783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6f2c7-98f6-4cf7-b971-28f169535943 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.022616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46436bf8ece14df0a01d637fecc57543'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.030839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46436bf8ece14df0a01d637fecc57543'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.035477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.036938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.037417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.037864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.046146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46436bf8ece14df0a01d637fecc57543'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.046866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46436bf8ece14df0a01d637fecc57543'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.048045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46436bf8ece14df0a01d637fecc57543'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.050051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.053394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.063425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1054/2081] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:26 2022] DELETE /identity/v3/users/46436bf8ece14df0a01d637fecc57543 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:27.063910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1028/2082] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:27.358806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.359058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.360510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:27.361319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:27.386877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15968c-895c-49de-aa20-b2670ebec256 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.457775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb2b9d-f947-44c7-897b-5cdda1263c8e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.464243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.464884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.465162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.465430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] PATH_INFO: `/v3/projects/05af09b5a814415480717d6a3facb4d7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.466725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] RBAC: Authorizing `identity:delete_project(project_id=05af09b5a814415480717d6a3facb4d7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.469163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.522532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.523212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.523513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.523786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] PATH_INFO: `/v3/projects/1f3f4603d9ae469daac3ab8e21cdcd9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.526363 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f3f4603d9ae469daac3ab8e21cdcd9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.528384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.540773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05af09b5a814415480717d6a3facb4d7', 'request_id': 'req-7507a063-1339-4bd3-bb12-1295ab57990e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.551579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05af09b5a814415480717d6a3facb4d7', 'request_id': 'req-7507a063-1339-4bd3-bb12-1295ab57990e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.567437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.575730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05af09b5a814415480717d6a3facb4d7', 'request_id': 'req-7507a063-1339-4bd3-bb12-1295ab57990e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.596761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.610228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] The token cache is being invalidate because project 05af09b5a814415480717d6a3facb4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:27.610517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.611195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7507a063-1339-4bd3-bb12-1295ab57990e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.617383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1055/2083] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:27 2022] DELETE /identity/v3/projects/05af09b5a814415480717d6a3facb4d7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:27.621791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca540-7b95-4c03-9ff5-bdc413418d00 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.622487 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca540-7b95-4c03-9ff5-bdc413418d00 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.623054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca540-7b95-4c03-9ff5-bdc413418d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.648350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3cdca540-7b95-4c03-9ff5-bdc413418d00 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:27.650158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3cdca540-7b95-4c03-9ff5-bdc413418d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:27.664445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f3f4603d9ae469daac3ab8e21cdcd9d', 'request_id': 'req-12a4e2dc-6515-4c0c-95ed-49182fc66b92'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.680555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f3f4603d9ae469daac3ab8e21cdcd9d', 'request_id': 'req-12a4e2dc-6515-4c0c-95ed-49182fc66b92'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.691887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f3f4603d9ae469daac3ab8e21cdcd9d', 'request_id': 'req-12a4e2dc-6515-4c0c-95ed-49182fc66b92'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.704207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] The token cache is being invalidate because project 1f3f4603d9ae469daac3ab8e21cdcd9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:27.704506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.705188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.707210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1029/2084] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:27 2022] DELETE /identity/v3/projects/1f3f4603d9ae469daac3ab8e21cdcd9d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:27.716838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.716838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:27.762822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1056/2085] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:27.772223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e533c-1436-444d-b75c-2214c44cbbb0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.772590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e533c-1436-444d-b75c-2214c44cbbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.772985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e533c-1436-444d-b75c-2214c44cbbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.813251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-490e533c-1436-444d-b75c-2214c44cbbb0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:27.814773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-490e533c-1436-444d-b75c-2214c44cbbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:27.828322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4e2dc-6515-4c0c-95ed-49182fc66b92 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.829002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1057/2086] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:27.830282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e533c-1436-444d-b75c-2214c44cbbb0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.834181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.834822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.835102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.835373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] PATH_INFO: `/v3/users/904eeea82220460db89b4694f90fbfc5` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.836742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=904eeea82220460db89b4694f90fbfc5)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.838666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.849397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e533c-1436-444d-b75c-2214c44cbbb0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.855997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.856974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.857312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.857715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.858815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:27.859335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.861207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.887572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1058/2087] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.887572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.887572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:27.918693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '904eeea82220460db89b4694f90fbfc5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.919707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '904eeea82220460db89b4694f90fbfc5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.929871 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '904eeea82220460db89b4694f90fbfc5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.951236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '904eeea82220460db89b4694f90fbfc5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.952154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '904eeea82220460db89b4694f90fbfc5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:27.964257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba1f7b-c71d-4455-8050-721269ec9383 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.968401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.969234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.969575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.969848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.972275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:27.980811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1030/2088] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:27 2022] DELETE /identity/v3/users/904eeea82220460db89b4694f90fbfc5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:27.981771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.993402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.032949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.102841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.104341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.108242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657976f-3706-4234-8283-d24cea736e07 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.116384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.117302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.117718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.118921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.119875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.123321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.126483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1059/2089] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:28.188255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1031/2090] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:27 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:28.189867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.208394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ae971-5e32-4c87-aa4d-6f8a19ee5323 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.245420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.304030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.304913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.305506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.306031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.306990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.310928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.345169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.347340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.356313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1060/2091] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:28 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:28.357969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.377459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b0771-2007-47bf-b19c-6d8fe5c0a830 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.385309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.385991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.386252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.386504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] PATH_INFO: `/v3/projects/0952490ef6754acdba5d2346ea566ad4` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.387666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0952490ef6754acdba5d2346ea566ad4)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.390174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.397489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde524bd-f76e-404f-89ad-f5ea415e0bef admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.409684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.410941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.412094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.414265 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.415201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.416952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.433448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1061/2092] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:28.435098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.448952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0952490ef6754acdba5d2346ea566ad4', 'request_id': 'req-8f218f5f-3b83-45e8-a6e4-2aba6073224d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:28.449156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd881a4f-e328-4f4b-a6b1-7f8737a3d410 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.467271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0952490ef6754acdba5d2346ea566ad4', 'request_id': 'req-8f218f5f-3b83-45e8-a6e4-2aba6073224d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:28.487582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0952490ef6754acdba5d2346ea566ad4', 'request_id': 'req-8f218f5f-3b83-45e8-a6e4-2aba6073224d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:28.501899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] The token cache is being invalidate because project 0952490ef6754acdba5d2346ea566ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:28.502200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:28.502915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8f218f5f-3b83-45e8-a6e4-2aba6073224d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:28.507903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1032/2093] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:28 2022] DELETE /identity/v3/projects/0952490ef6754acdba5d2346ea566ad4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:28.510662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4877dd-cac2-4f29-b05d-3c9128fc9e36 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.511142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4877dd-cac2-4f29-b05d-3c9128fc9e36 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.511788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4877dd-cac2-4f29-b05d-3c9128fc9e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.525689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.526304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.526582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.526859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.527499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.529565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.534980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7d4877dd-cac2-4f29-b05d-3c9128fc9e36 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:28.537258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7d4877dd-cac2-4f29-b05d-3c9128fc9e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.543728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1062/2094] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:28.545340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.548132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.560193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1033/2095] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:28.568412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ad0f7-243a-4ef5-910b-57a83ecc8490 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.568697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ad0f7-243a-4ef5-910b-57a83ecc8490 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.568968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ad0f7-243a-4ef5-910b-57a83ecc8490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.572657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e786-8f6c-4681-b6f7-303ec654b28b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.578350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e95efa-4965-4336-92a0-ca0559966b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.579077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e95efa-4965-4336-92a0-ca0559966b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.589535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e95efa-4965-4336-92a0-ca0559966b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.589837 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e95efa-4965-4336-92a0-ca0559966b48 admin admin] PATH_INFO: `/v3/projects/437d81014c184774b962e3a70e0a8427/users/a779903d929249c9bb5c1c3d47f6e2d6/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.591018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-258ad0f7-243a-4ef5-910b-57a83ecc8490 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:28.592443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-258ad0f7-243a-4ef5-910b-57a83ecc8490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.602682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1034/2096] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:28.605992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ad0f7-243a-4ef5-910b-57a83ecc8490 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.615886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ad0f7-243a-4ef5-910b-57a83ecc8490 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.620907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.621514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.621793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.622062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.622705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:28.623007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.628134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e95efa-4965-4336-92a0-ca0559966b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437d81014c184774b962e3a70e0a8427, user_id=a779903d929249c9bb5c1c3d47f6e2d6, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.633619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e95efa-4965-4336-92a0-ca0559966b48 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.636858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.652557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1063/2097] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:28 2022] PUT /identity/v3/projects/437d81014c184774b962e3a70e0a8427/users/a779903d929249c9bb5c1c3d47f6e2d6/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:28.662773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7791fef-77d3-477a-af1e-717ba7ec3390 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.663839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7791fef-77d3-477a-af1e-717ba7ec3390 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.664324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7791fef-77d3-477a-af1e-717ba7ec3390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.666432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1035/2098] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:28.671052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.699792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482440-fbfe-4793-af3d-d9ca9ea5b7a2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.721748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c7791fef-77d3-477a-af1e-717ba7ec3390 None None] MFA Rules not processed for user `a779903d929249c9bb5c1c3d47f6e2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:28.722965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c7791fef-77d3-477a-af1e-717ba7ec3390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.800565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.802923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.803366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.804185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.805570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.808136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.828373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.857588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1064/2099] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:28.867944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1036/2100] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:28 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:28.869093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7791fef-77d3-477a-af1e-717ba7ec3390 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.876634 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.877723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.878954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.889835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7791fef-77d3-477a-af1e-717ba7ec3390 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.965344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8ee7f-6925-49ef-8ba0-2aab51bcfcd5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.971179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.972090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.972441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.972880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.988631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.990122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.990701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.991203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.993094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.996035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.038684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1065/2101] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:28 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:29.043645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.063215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747307db-1b11-44b7-9558-51624c21703c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.070005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.070681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.071156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.071433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.072156 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.074651 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.077567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.079654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.089812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1066/2102] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:29.093383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.105946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1037/2103] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:29.108595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce7ddc-ee3e-4dbb-b929-cd3bc79cd9ca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.114542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.116136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.116136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.116136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.117026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.118896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.132881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.148020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1067/2104] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:29.152367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.154523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f14d7-ab37-4a86-9962-983c468a1c3c service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.160635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.163541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.163541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.163824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] PATH_INFO: `/v3/users/a779903d929249c9bb5c1c3d47f6e2d6` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.164706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ff69b-7c7c-4606-8024-6403728dd5b3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.165158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a779903d929249c9bb5c1c3d47f6e2d6)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.167478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.169650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8f681-7af3-4b47-bac3-526082a69c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.170684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8f681-7af3-4b47-bac3-526082a69c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.170968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8f681-7af3-4b47-bac3-526082a69c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.171553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8f681-7af3-4b47-bac3-526082a69c81 admin admin] PATH_INFO: `/v3/projects/e4cbb840c67c4bbaa6bad03eecdb7cdc/users/62802bf56af64d6cbe51c8179b311b87/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.189905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8f681-7af3-4b47-bac3-526082a69c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4cbb840c67c4bbaa6bad03eecdb7cdc, user_id=62802bf56af64d6cbe51c8179b311b87, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.195261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8f681-7af3-4b47-bac3-526082a69c81 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.214761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1068/2105] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:29 2022] PUT /identity/v3/projects/e4cbb840c67c4bbaa6bad03eecdb7cdc/users/62802bf56af64d6cbe51c8179b311b87/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:29.226695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49d706-0c10-4b13-838e-a1e66f6818b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.227177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49d706-0c10-4b13-838e-a1e66f6818b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.227638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49d706-0c10-4b13-838e-a1e66f6818b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.264358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a779903d929249c9bb5c1c3d47f6e2d6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:29.265193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a779903d929249c9bb5c1c3d47f6e2d6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:29.273776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5b49d706-0c10-4b13-838e-a1e66f6818b5 None None] MFA Rules not processed for user `62802bf56af64d6cbe51c8179b311b87`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:29.275267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a779903d929249c9bb5c1c3d47f6e2d6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:29.276044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5b49d706-0c10-4b13-838e-a1e66f6818b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:29.292359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a779903d929249c9bb5c1c3d47f6e2d6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:29.293158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a779903d929249c9bb5c1c3d47f6e2d6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:29.312513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1038/2106] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:29 2022] DELETE /identity/v3/users/a779903d929249c9bb5c1c3d47f6e2d6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:29.313318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.332004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:29.359989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1069/2107] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:29.361354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.361642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.361943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.365471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.389385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:29.390918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:29.411111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71215-bb46-46ba-a0e9-b1bfa4ace976 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.414904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.415534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.415819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.416120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.417859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:29.425681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.463481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1070/2108] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:29.464267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.465550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:29.487668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.489133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.499391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1039/2109] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:29.500941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.502360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:29.543994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498438be-8f2b-4d24-9b54-a3fb55992882 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.550169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa583d6f-8a88-4472-aaae-c07e2abce106 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.550847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583d6f-8a88-4472-aaae-c07e2abce106 service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.551459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583d6f-8a88-4472-aaae-c07e2abce106 service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.551764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583d6f-8a88-4472-aaae-c07e2abce106 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.565553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766d017-461a-4e0e-bbb5-f422a73007d1 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.573288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.573915 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.574195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.574467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.585017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.586552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.597442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1040/2110] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:29.628147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa583d6f-8a88-4472-aaae-c07e2abce106 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.629892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa583d6f-8a88-4472-aaae-c07e2abce106 service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.641335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1071/2111] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:30.159816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.165442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:30.192538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477c8d9-a9e6-43c7-b01f-4c7737d59b80 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.201568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.202216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.202497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.202772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] PATH_INFO: `/v3/projects/437d81014c184774b962e3a70e0a8427` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.204007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437d81014c184774b962e3a70e0a8427)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.205980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.256701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437d81014c184774b962e3a70e0a8427', 'request_id': 'req-0d340cca-9eb2-4f99-98af-ec87a4189460'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:30.263005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437d81014c184774b962e3a70e0a8427', 'request_id': 'req-0d340cca-9eb2-4f99-98af-ec87a4189460'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:30.272959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437d81014c184774b962e3a70e0a8427', 'request_id': 'req-0d340cca-9eb2-4f99-98af-ec87a4189460'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:30.277251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.285682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] The token cache is being invalidate because project 437d81014c184774b962e3a70e0a8427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:30.285988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:30.287730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0d340cca-9eb2-4f99-98af-ec87a4189460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:30.289641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1041/2112] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:30 2022] DELETE /identity/v3/projects/437d81014c184774b962e3a70e0a8427 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:30.294632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516fae-f0a6-4401-a307-605dd02778cc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.295102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516fae-f0a6-4401-a307-605dd02778cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.295306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516fae-f0a6-4401-a307-605dd02778cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.315846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-62516fae-f0a6-4401-a307-605dd02778cc None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:30.317096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62516fae-f0a6-4401-a307-605dd02778cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:30.385939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1072/2113] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:30.393721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10f89-d5ba-40fb-a3a7-48c01f99cc59 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.394008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10f89-d5ba-40fb-a3a7-48c01f99cc59 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.394283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10f89-d5ba-40fb-a3a7-48c01f99cc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.415150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d5a10f89-d5ba-40fb-a3a7-48c01f99cc59 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:30.416409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d5a10f89-d5ba-40fb-a3a7-48c01f99cc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:30.425544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1042/2114] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:30.430411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62516fae-f0a6-4401-a307-605dd02778cc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.440693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62516fae-f0a6-4401-a307-605dd02778cc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.446067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.446681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.446956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.447224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.448129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:30.448485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.450141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.458145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.468374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1073/2115] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:30.469534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10f89-d5ba-40fb-a3a7-48c01f99cc59 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.479117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10f89-d5ba-40fb-a3a7-48c01f99cc59 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.483785 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.484576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.484861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.485135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.485784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.487780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.518790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1043/2116] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:30 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:30.522766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.532701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f9177-e595-47f4-a71d-c3e89e91607b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.592965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.593669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.594064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.594393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.595255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.597694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.624639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1074/2117] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:30 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:30.625594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.639387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49024aef-65ee-47d3-bb24-ba458dcad33f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.644782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.645457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.645747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.646027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.646671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.648307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.664374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1044/2118] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:30.665320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.679706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807854-cd35-4969-93ae-9886e1b0088c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.685315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dd464-c9c6-4a9e-9c4f-e8cc3539ff25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.686053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd464-c9c6-4a9e-9c4f-e8cc3539ff25 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.686354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd464-c9c6-4a9e-9c4f-e8cc3539ff25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.686914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd464-c9c6-4a9e-9c4f-e8cc3539ff25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.695943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dd464-c9c6-4a9e-9c4f-e8cc3539ff25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.695943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dd464-c9c6-4a9e-9c4f-e8cc3539ff25 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.721182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1075/2119] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:30.725454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.743336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd331ccd-9141-4428-bf96-f233de6ddab6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.748266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.748934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.749221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.749540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] PATH_INFO: `/v3/projects/373d3f9a0a12420aa64df6d3e0c8312d/users/536c4839acf44b03bc316b906911347b/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.766999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=373d3f9a0a12420aa64df6d3e0c8312d, user_id=536c4839acf44b03bc316b906911347b, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.769922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.785982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1045/2120] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:30 2022] PUT /identity/v3/projects/373d3f9a0a12420aa64df6d3e0c8312d/users/536c4839acf44b03bc316b906911347b/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:30.797529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.797816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.798085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.827924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] MFA Rules not processed for user `536c4839acf44b03bc316b906911347b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:30.833748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:30.918600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1076/2121] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:30.931231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.932992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:31.015329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0d65-098e-400e-aa56-a9c119234b10 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.019063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d013e-91c7-412e-b0c2-a6b8c4625eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.019651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d013e-91c7-412e-b0c2-a6b8c4625eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.019933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d013e-91c7-412e-b0c2-a6b8c4625eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.020208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d013e-91c7-412e-b0c2-a6b8c4625eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.029521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d013e-91c7-412e-b0c2-a6b8c4625eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.030985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d013e-91c7-412e-b0c2-a6b8c4625eb7 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.045923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1046/2122] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:33.593487 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.595115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:33.665554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94e2aa-24af-4d8d-8f79-93ad88c43f71 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.670400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.671085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.671380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.671697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] PATH_INFO: `/v3/users/9b2bf733b0264aa29c6ddc37acee1e0e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.672840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2bf733b0264aa29c6ddc37acee1e0e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:33.674821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:33.681228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:33.747796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2bf733b0264aa29c6ddc37acee1e0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:33.754927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2bf733b0264aa29c6ddc37acee1e0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:33.767229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2bf733b0264aa29c6ddc37acee1e0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:33.768153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2bf733b0264aa29c6ddc37acee1e0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:33.768644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2bf733b0264aa29c6ddc37acee1e0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:33.785994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1077/2123] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:33 2022] DELETE /identity/v3/users/9b2bf733b0264aa29c6ddc37acee1e0e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:33.791359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.791873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.791964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.813090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:33.814836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:33.887872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1047/2124] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:33.897998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.899422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:33.961293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5d916-a6e9-42da-8fc4-aee951a2810e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.964962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.965573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.965850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.966120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.975385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:33.976871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:33.986809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1078/2125] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:34.315906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.317953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:34.334437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea77706-a7e8-4301-8a09-9e7c72d76bf6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:34.341704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:34.342671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.342950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.343178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] PATH_INFO: `/v3/projects/2c46f2aef6d34466952e40343ab9ad17` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.345163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c46f2aef6d34466952e40343ab9ad17)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.347428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:34.390236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:34.413602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c46f2aef6d34466952e40343ab9ad17', 'request_id': 'req-4cc29061-c5db-4f8a-a634-306673a3223b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.426946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c46f2aef6d34466952e40343ab9ad17', 'request_id': 'req-4cc29061-c5db-4f8a-a634-306673a3223b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.436027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c46f2aef6d34466952e40343ab9ad17', 'request_id': 'req-4cc29061-c5db-4f8a-a634-306673a3223b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.448816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] The token cache is being invalidate because project 2c46f2aef6d34466952e40343ab9ad17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:34.449116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.449761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.451252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1048/2126] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:34 2022] DELETE /identity/v3/projects/2c46f2aef6d34466952e40343ab9ad17 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:34.455357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.465221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de4a8-c122-4545-95f9-ab9f9da69272 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:34.524802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:34.525443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.525725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.525997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] PATH_INFO: `/v3/users/078e1dcc9cfe4d0595770bcfa75bf0b4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.527123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=078e1dcc9cfe4d0595770bcfa75bf0b4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.529144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:34.565830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:34.580827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078e1dcc9cfe4d0595770bcfa75bf0b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.587824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078e1dcc9cfe4d0595770bcfa75bf0b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.600204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078e1dcc9cfe4d0595770bcfa75bf0b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.600908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078e1dcc9cfe4d0595770bcfa75bf0b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.601516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078e1dcc9cfe4d0595770bcfa75bf0b4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:34.617301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1079/2127] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:34 2022] DELETE /identity/v3/users/078e1dcc9cfe4d0595770bcfa75bf0b4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:34.894828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.896361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:34.915655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc29061-c5db-4f8a-a634-306673a3223b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:34.977130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:34.977791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.978078 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.978755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] PATH_INFO: `/v3/projects/5f9b6d607097418cbbee932d992a1ea9` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.980004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9b6d607097418cbbee932d992a1ea9)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.981932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.029669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9b6d607097418cbbee932d992a1ea9', 'request_id': 'req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.035660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9b6d607097418cbbee932d992a1ea9', 'request_id': 'req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.043361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9b6d607097418cbbee932d992a1ea9', 'request_id': 'req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.054459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] The token cache is being invalidate because project 5f9b6d607097418cbbee932d992a1ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:35.054753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.055378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.056895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1049/2128] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:34 2022] DELETE /identity/v3/projects/5f9b6d607097418cbbee932d992a1ea9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:35.060978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.062436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:35.075735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a233c9-787a-4276-8072-f254dccdf3e9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.136751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.137637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.138010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.138287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] PATH_INFO: `/v3/users/5c7a2a82e86d468780fa14a61c7eb33e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.139709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7a2a82e86d468780fa14a61c7eb33e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.142193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.205538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7a2a82e86d468780fa14a61c7eb33e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.213325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7a2a82e86d468780fa14a61c7eb33e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.226107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7a2a82e86d468780fa14a61c7eb33e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.226823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7a2a82e86d468780fa14a61c7eb33e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.227431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8abd8080-d91a-40ee-acab-5f8f14f3f03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7a2a82e86d468780fa14a61c7eb33e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.261217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1080/2129] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:35 2022] DELETE /identity/v3/users/5c7a2a82e86d468780fa14a61c7eb33e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:35.556570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.558118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:35.571179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0fe8c-26c7-424e-b2c5-260c36fa9139 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.620558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.636376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.637045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.637439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.637725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] PATH_INFO: `/v3/projects/8c720c14c32a4ff792b1548215df8e1c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.638896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c720c14c32a4ff792b1548215df8e1c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.640838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.690819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c720c14c32a4ff792b1548215df8e1c', 'request_id': 'req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.697696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c720c14c32a4ff792b1548215df8e1c', 'request_id': 'req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.705608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c720c14c32a4ff792b1548215df8e1c', 'request_id': 'req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.716133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] The token cache is being invalidate because project 8c720c14c32a4ff792b1548215df8e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:35.716551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.717189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ffad9cfe-e51a-421d-ae81-cc7d42faafb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:35.718717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1050/2130] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:35 2022] DELETE /identity/v3/projects/8c720c14c32a4ff792b1548215df8e1c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:35.723729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a794-c9b3-4527-8852-aa81acc401e5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.724176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a794-c9b3-4527-8852-aa81acc401e5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.724293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a794-c9b3-4527-8852-aa81acc401e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.744128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-27f2a794-c9b3-4527-8852-aa81acc401e5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:35.745327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-27f2a794-c9b3-4527-8852-aa81acc401e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:35.766357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.807974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1081/2131] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:35.816118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca25fd-5135-4c01-80c2-98b1ecad7014 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.816413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca25fd-5135-4c01-80c2-98b1ecad7014 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.816690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca25fd-5135-4c01-80c2-98b1ecad7014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.836878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ddca25fd-5135-4c01-80c2-98b1ecad7014 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:35.838094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ddca25fd-5135-4c01-80c2-98b1ecad7014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:35.845939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1051/2132] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:35.850699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a794-c9b3-4527-8852-aa81acc401e5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.860519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a794-c9b3-4527-8852-aa81acc401e5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.865854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.866536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.866820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.867118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.867794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:35.868095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.869744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.881697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1082/2133] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:35.885896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca25fd-5135-4c01-80c2-98b1ecad7014 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.895539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca25fd-5135-4c01-80c2-98b1ecad7014 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.900860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.901457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.901737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.902011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.902641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.904596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.932670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1052/2134] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:35 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:35.937076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.947649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb0537-1d48-4638-bd41-f0974bb0c93b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.002855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.003459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.003785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.004056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.004682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.007082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.027690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1083/2135] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:35 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:36.031895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.043280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba6c7d-b073-4a61-a011-bd174f711aea admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.049690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.050391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.050819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.051042 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.051980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.053576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.064383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1053/2136] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:36.069728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.079992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb028f7-78c3-4361-8197-8bf623f8ea77 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.085358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d2fac-cd45-4960-9d81-93247c3b7154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.085956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2fac-cd45-4960-9d81-93247c3b7154 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.086236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2fac-cd45-4960-9d81-93247c3b7154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.086509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2fac-cd45-4960-9d81-93247c3b7154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.087145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d2fac-cd45-4960-9d81-93247c3b7154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.088775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d2fac-cd45-4960-9d81-93247c3b7154 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.099800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1084/2137] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:36.104142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.114429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427fcd8-f29a-4049-9a01-e3b377953025 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.119052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.119726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.120005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.120283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] PATH_INFO: `/v3/projects/ffa05efb090b4e9199ce821347d00e57/users/081ead72315e4b58bea1785367c0c6f5/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.135106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa05efb090b4e9199ce821347d00e57, user_id=081ead72315e4b58bea1785367c0c6f5, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.137757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.151198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1054/2138] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:36 2022] PUT /identity/v3/projects/ffa05efb090b4e9199ce821347d00e57/users/081ead72315e4b58bea1785367c0c6f5/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:36.165061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.165331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.165629 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.194595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] MFA Rules not processed for user `081ead72315e4b58bea1785367c0c6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:36.195881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:36.268635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1085/2139] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:36.282094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.284288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:36.365543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae370e5b-0273-4d50-99dc-d9976031496b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.374606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcccb701-f97c-4608-a5ca-a3af5e9df9e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.375508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccb701-f97c-4608-a5ca-a3af5e9df9e9 service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.375877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccb701-f97c-4608-a5ca-a3af5e9df9e9 service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.376225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccb701-f97c-4608-a5ca-a3af5e9df9e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.387562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccb701-f97c-4608-a5ca-a3af5e9df9e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.389184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccb701-f97c-4608-a5ca-a3af5e9df9e9 service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.399360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1055/2140] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:39.744024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:39.745534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:39.776015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:39.834468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90f38c-2d18-4d75-9748-a99d4a6a85a1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:39.839355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:39.840028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:39.840360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:39.840575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] PATH_INFO: `/v3/users/536c4839acf44b03bc316b906911347b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:39.841703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=536c4839acf44b03bc316b906911347b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:39.843727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:39.891467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536c4839acf44b03bc316b906911347b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:39.899179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536c4839acf44b03bc316b906911347b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:39.912635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536c4839acf44b03bc316b906911347b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:39.913575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536c4839acf44b03bc316b906911347b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:39.914339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536c4839acf44b03bc316b906911347b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:39.928480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1086/2141] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:39 2022] DELETE /identity/v3/users/536c4839acf44b03bc316b906911347b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:39.932923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:39.933509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:39.933509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:39.953842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:39.955082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:39.964360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:40.022200 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1056/2142] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:40.031973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.033670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:40.103113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73423883-84b7-40ed-8b75-8b8498955f94 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:40.120474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f27c8-0ca0-4435-b18f-f92924728728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:40.121126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f27c8-0ca0-4435-b18f-f92924728728 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.121439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f27c8-0ca0-4435-b18f-f92924728728 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.121718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f27c8-0ca0-4435-b18f-f92924728728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.139314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f27c8-0ca0-4435-b18f-f92924728728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:40.141640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f27c8-0ca0-4435-b18f-f92924728728 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:40.167590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1087/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:40.571736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.573214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:40.591650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73054be-4509-47ae-ace0-f8b62ffc641b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:40.597269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:40.597827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.598135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.598404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] PATH_INFO: `/v3/projects/373d3f9a0a12420aa64df6d3e0c8312d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.600275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=373d3f9a0a12420aa64df6d3e0c8312d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:40.602510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:40.663054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373d3f9a0a12420aa64df6d3e0c8312d', 'request_id': 'req-7de88a06-056d-4174-b65a-e6fd96a82d35'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:40.670614 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373d3f9a0a12420aa64df6d3e0c8312d', 'request_id': 'req-7de88a06-056d-4174-b65a-e6fd96a82d35'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:40.678481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373d3f9a0a12420aa64df6d3e0c8312d', 'request_id': 'req-7de88a06-056d-4174-b65a-e6fd96a82d35'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:40.689366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] The token cache is being invalidate because project 373d3f9a0a12420aa64df6d3e0c8312d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:40.689650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:40.690287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7de88a06-056d-4174-b65a-e6fd96a82d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:40.692057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1057/2144] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:40 2022] DELETE /identity/v3/projects/373d3f9a0a12420aa64df6d3e0c8312d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:40.697018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e1f89-4b58-43c0-b496-4366246b605c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.697310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e1f89-4b58-43c0-b496-4366246b605c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.698355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e1f89-4b58-43c0-b496-4366246b605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.725771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f49e1f89-4b58-43c0-b496-4366246b605c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:40.727099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f49e1f89-4b58-43c0-b496-4366246b605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:40.778488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:40.799069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1088/2145] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:40.807738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d02436-7041-48f9-a329-b3d4ef40fdc3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.808099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d02436-7041-48f9-a329-b3d4ef40fdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.808432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d02436-7041-48f9-a329-b3d4ef40fdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.828810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-67d02436-7041-48f9-a329-b3d4ef40fdc3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:40.830076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-67d02436-7041-48f9-a329-b3d4ef40fdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:40.839291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1058/2146] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:40.844973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e1f89-4b58-43c0-b496-4366246b605c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.855772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e1f89-4b58-43c0-b496-4366246b605c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:40.861364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:40.863601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.864163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.864446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.865115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:40.865414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:40.867025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:40.882475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1089/2147] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:40.884592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d02436-7041-48f9-a329-b3d4ef40fdc3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.894866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d02436-7041-48f9-a329-b3d4ef40fdc3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:40.900803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:40.901435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.901760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.902174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.902747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:40.904755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:40.940408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1059/2148] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:40 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:40.948361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.962819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1629ab4-fbe8-4eb7-8848-e6f0c7155527 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.026773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.027630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.028078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.028503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.029304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.031429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.061031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1090/2149] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:40 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:41.065720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.077330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44994b-407d-411d-bc6e-16426825ae96 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.083749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.084395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.084723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.085040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.085725 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.087515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.098591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1060/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:41.103333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.115536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6cb1f-57d0-4f55-993f-c59d3d412eae admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.121707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c6d52-5407-4165-8a2c-cca699a97748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.124345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c6d52-5407-4165-8a2c-cca699a97748 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.124345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c6d52-5407-4165-8a2c-cca699a97748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.124345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c6d52-5407-4165-8a2c-cca699a97748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.124345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c6d52-5407-4165-8a2c-cca699a97748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.137941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c6d52-5407-4165-8a2c-cca699a97748 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.158303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1091/2151] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:41.162331 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.176734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39300f-9c92-4d34-8728-ae4339e6068e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.186971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.188171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.188695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.189027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] PATH_INFO: `/v3/projects/e572c089027946768e92e19f26e36191/users/c8ff83b4100241df8f5bd7c925c107b2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.210987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e572c089027946768e92e19f26e36191, user_id=c8ff83b4100241df8f5bd7c925c107b2, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.214409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.232723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1061/2152] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:41 2022] PUT /identity/v3/projects/e572c089027946768e92e19f26e36191/users/c8ff83b4100241df8f5bd7c925c107b2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:41.242711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43c613-9ad6-4213-94b9-b071ed3d6d3e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.243000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43c613-9ad6-4213-94b9-b071ed3d6d3e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.243272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43c613-9ad6-4213-94b9-b071ed3d6d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.273600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3e43c613-9ad6-4213-94b9-b071ed3d6d3e None None] MFA Rules not processed for user `c8ff83b4100241df8f5bd7c925c107b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:41.273801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3e43c613-9ad6-4213-94b9-b071ed3d6d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:41.364013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1092/2153] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:41.369017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.418553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.441423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e806-cfe5-4e3f-9900-5c707847a804 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.455875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.455875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.455875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.455875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.456238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.456238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.490541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1062/2154] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:41 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:41.494323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43c613-9ad6-4213-94b9-b071ed3d6d3e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.510920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43c613-9ad6-4213-94b9-b071ed3d6d3e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.562937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.592465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.594048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.594518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.596171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.597685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.600142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.643411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.644373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1093/2155] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:41 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:41.658971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5213ea-fc6f-44b7-b00f-d9d62fee0787 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.664472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.665090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.665431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.665781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.672215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.673780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.694598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1063/2156] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:41.697253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.708155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652a084-2d2b-483a-b5b1-09e803386646 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.717187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.717955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.718392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.718848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] PATH_INFO: `/v3/projects/28bda0ad1af04c87b86f49e58adccdf3/users/bf049488f4e04ba3a081409589fd5140/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.738282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bda0ad1af04c87b86f49e58adccdf3, user_id=bf049488f4e04ba3a081409589fd5140, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.741849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.757157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1094/2157] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:41 2022] PUT /identity/v3/projects/28bda0ad1af04c87b86f49e58adccdf3/users/bf049488f4e04ba3a081409589fd5140/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:41.760985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.771219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0416bd-79ae-4e5a-b48a-fbed477dccf8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.832953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdea819-ab85-482d-971b-7224fc6b594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.833624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea819-ab85-482d-971b-7224fc6b594c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.833949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea819-ab85-482d-971b-7224fc6b594c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.834269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea819-ab85-482d-971b-7224fc6b594c admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.834944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdea819-ab85-482d-971b-7224fc6b594c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.836616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdea819-ab85-482d-971b-7224fc6b594c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.850340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1064/2158] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:41.851640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.863172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97affc1e-bb1e-42fe-9d8e-3dc5c2fe2a2d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.868872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.869517 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.869794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.870064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] PATH_INFO: `/v3/projects/28bda0ad1af04c87b86f49e58adccdf3/users/bf049488f4e04ba3a081409589fd5140/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.871995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bda0ad1af04c87b86f49e58adccdf3, user_id=bf049488f4e04ba3a081409589fd5140, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.874618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.885164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1095/2159] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:41 2022] PUT /identity/v3/projects/28bda0ad1af04c87b86f49e58adccdf3/users/bf049488f4e04ba3a081409589fd5140/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:41.896374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.896638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.896958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.934792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] MFA Rules not processed for user `bf049488f4e04ba3a081409589fd5140`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:41.936538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:42.028458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1065/2160] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:42.040933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.042947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:42.125078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1b6ce-be35-4b98-aae2-9fec7b1327c2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.131287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.136830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.137101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.137859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.244200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.246264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.272731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1096/2161] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:44.582369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:44.584158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:44.635232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:44.662342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fa8f2-7c10-49a2-bfe7-9ee0675b6fb9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:44.667880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:44.668461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:44.668787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:44.669083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] PATH_INFO: `/v3/users/56a554beedec4915b6ad1066823c8d4e` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:44.670287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a554beedec4915b6ad1066823c8d4e)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:44.672277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:44.722856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a554beedec4915b6ad1066823c8d4e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:44.723601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a554beedec4915b6ad1066823c8d4e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:44.731468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a554beedec4915b6ad1066823c8d4e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:44.744355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a554beedec4915b6ad1066823c8d4e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:44.745048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-45695362-f2cd-4038-96c1-041399fa2beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a554beedec4915b6ad1066823c8d4e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:44.757404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1066/2162] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:44 2022] DELETE /identity/v3/users/56a554beedec4915b6ad1066823c8d4e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:44.937547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:44.949310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a986d2-60d8-4c2c-8e23-e5633de96623 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.014413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:45.033902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.034683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.035881 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.036764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] PATH_INFO: `/v3/projects/e966946c8e5a4e71803e8e37971160e1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.038970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e966946c8e5a4e71803e8e37971160e1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.041895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.099413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e966946c8e5a4e71803e8e37971160e1', 'request_id': 'req-9ebcd481-d2d5-4c65-9491-7842853083ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.106875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e966946c8e5a4e71803e8e37971160e1', 'request_id': 'req-9ebcd481-d2d5-4c65-9491-7842853083ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.113365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e966946c8e5a4e71803e8e37971160e1', 'request_id': 'req-9ebcd481-d2d5-4c65-9491-7842853083ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.128800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] The token cache is being invalidate because project e966946c8e5a4e71803e8e37971160e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:45.129101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.130043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.135160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1097/2163] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:44 2022] DELETE /identity/v3/projects/e966946c8e5a4e71803e8e37971160e1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:45.136971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecc8f7-7228-4b7d-b128-6581bc7b4e74 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.137453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecc8f7-7228-4b7d-b128-6581bc7b4e74 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.137869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecc8f7-7228-4b7d-b128-6581bc7b4e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.160083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b1ecc8f7-7228-4b7d-b128-6581bc7b4e74 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:45.162056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b1ecc8f7-7228-4b7d-b128-6581bc7b4e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.238760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.240333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.287943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1067/2164] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:45.299637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c562f-b441-457d-8b5f-fd8b50d996e4 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.299904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c562f-b441-457d-8b5f-fd8b50d996e4 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.300177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c562f-b441-457d-8b5f-fd8b50d996e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.323763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5c6c562f-b441-457d-8b5f-fd8b50d996e4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:45.325584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5c6c562f-b441-457d-8b5f-fd8b50d996e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.337563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1068/2165] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:45.342738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c562f-b441-457d-8b5f-fd8b50d996e4 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.350724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcd481-d2d5-4c65-9491-7842853083ae admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.353250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c562f-b441-457d-8b5f-fd8b50d996e4 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.355650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.356297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.356582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.356860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] PATH_INFO: `/v3/users/c8ff83b4100241df8f5bd7c925c107b2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.357904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.358039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ff83b4100241df8f5bd7c925c107b2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.358504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.358784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.359056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.359727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:45.359975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.360092 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.361645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.373418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1069/2166] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:45.382653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.399253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d2f95-bdd4-4931-a86b-7a154b62b105 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.407314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.408210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.408497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.408815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.409458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.411977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.419563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ff83b4100241df8f5bd7c925c107b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.427116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ff83b4100241df8f5bd7c925c107b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.440242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ff83b4100241df8f5bd7c925c107b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.440955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ff83b4100241df8f5bd7c925c107b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.441565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-009a5403-b0a3-4dcd-9ac0-de69fa1d528e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ff83b4100241df8f5bd7c925c107b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:45.444083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1070/2167] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:45.448353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:45.448522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.450732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.464821 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1098/2168] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:45 2022] DELETE /identity/v3/users/c8ff83b4100241df8f5bd7c925c107b2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:45.465370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:45.469599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.470391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.471761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.509304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:45.510523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.556683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b23d3-f722-435a-8cc2-242941a97882 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.591119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1099/2169] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:45.598356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.599857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.624853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.625509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.625796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.626070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.626704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.631188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.652713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1071/2170] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:45.656799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.659404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c79da9-c655-49f2-bfb4-c44613e52312 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.663118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.663761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.664043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.664317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.666661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65c25-d498-4cf4-846a-9a6f4ecbf647 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.672009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.672597 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.672883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.673160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.673279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.673791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.674631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.675534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.684488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1100/2171] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:03:45.689186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1072/2172] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:45.690449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.701476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f1e3-8261-49f3-bb7b-8f1cad50a89b service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.706631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765f189-82e6-4252-ab25-e5d63c6e31b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.707277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765f189-82e6-4252-ab25-e5d63c6e31b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.707556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765f189-82e6-4252-ab25-e5d63c6e31b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.708472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765f189-82e6-4252-ab25-e5d63c6e31b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.709255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765f189-82e6-4252-ab25-e5d63c6e31b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.715559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765f189-82e6-4252-ab25-e5d63c6e31b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.725988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1101/2173] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:45.727820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.738229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c149d32-3c59-421b-811e-c90f9f84ccb1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.744099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.744820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.745105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.745412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] PATH_INFO: `/v3/projects/abb14f47b1654c99ae6989f276bc629c/users/1926f8ca5b9e4cb1a721a71510e52404/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.763107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb14f47b1654c99ae6989f276bc629c, user_id=1926f8ca5b9e4cb1a721a71510e52404, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.765828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.778857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1073/2174] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:45 2022] PUT /identity/v3/projects/abb14f47b1654c99ae6989f276bc629c/users/1926f8ca5b9e4cb1a721a71510e52404/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:45.789213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.789489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.789740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.822253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] MFA Rules not processed for user `1926f8ca5b9e4cb1a721a71510e52404`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:45.823617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.899189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1102/2175] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:45.911508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.913295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.978643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.981190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:46.004157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.006290 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696605-a660-45e3-9748-ca3d81f63a13 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.016378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.017003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.017289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.017618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.024701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.028076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.031083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.040831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1074/2176] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:46.061587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f458ab-0989-420a-b510-fed34ff8a3ed None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.067993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.068913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.069208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.069593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] PATH_INFO: `/v3/projects/e572c089027946768e92e19f26e36191` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.074852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e572c089027946768e92e19f26e36191)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.074852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.124878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e572c089027946768e92e19f26e36191', 'request_id': 'req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.131254 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e572c089027946768e92e19f26e36191', 'request_id': 'req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.136971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e572c089027946768e92e19f26e36191', 'request_id': 'req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.151901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] The token cache is being invalidate because project e572c089027946768e92e19f26e36191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:46.152102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.152903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.154637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1103/2177] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:45 2022] DELETE /identity/v3/projects/e572c089027946768e92e19f26e36191 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:46.158796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.169867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d89104-6db8-4bd9-a213-f9f8aa675989 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.231710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.232341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.232616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.232890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] PATH_INFO: `/v3/users/bf049488f4e04ba3a081409589fd5140` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.234033 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf049488f4e04ba3a081409589fd5140)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.235958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.288527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf049488f4e04ba3a081409589fd5140'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.289287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf049488f4e04ba3a081409589fd5140'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.297024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf049488f4e04ba3a081409589fd5140'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.313613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf049488f4e04ba3a081409589fd5140'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.314745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da2d7e7f-8b55-4692-81c3-0c0bceab5e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf049488f4e04ba3a081409589fd5140'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.338880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1075/2178] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:46 2022] DELETE /identity/v3/users/bf049488f4e04ba3a081409589fd5140 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:46.738678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.740991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:46.754286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5d6f6-bf7e-4831-8964-6a4f7ab3fc81 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.811776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.812443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.812723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.812996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] PATH_INFO: `/v3/projects/28bda0ad1af04c87b86f49e58adccdf3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.814121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bda0ad1af04c87b86f49e58adccdf3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.816168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.861170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bda0ad1af04c87b86f49e58adccdf3', 'request_id': 'req-68acca2c-d092-41e6-9acf-200609e1d257'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.862268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.869135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bda0ad1af04c87b86f49e58adccdf3', 'request_id': 'req-68acca2c-d092-41e6-9acf-200609e1d257'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.875448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bda0ad1af04c87b86f49e58adccdf3', 'request_id': 'req-68acca2c-d092-41e6-9acf-200609e1d257'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.887355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] The token cache is being invalidate because project 28bda0ad1af04c87b86f49e58adccdf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:46.887767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.888395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68acca2c-d092-41e6-9acf-200609e1d257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:46.890091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1104/2179] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:46 2022] DELETE /identity/v3/projects/28bda0ad1af04c87b86f49e58adccdf3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:46.897891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b050-ad74-49a4-8268-a5b59605858d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.898631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b050-ad74-49a4-8268-a5b59605858d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.898631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b050-ad74-49a4-8268-a5b59605858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.918039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bed8b050-ad74-49a4-8268-a5b59605858d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:46.919224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bed8b050-ad74-49a4-8268-a5b59605858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:46.980032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1076/2180] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:46.988204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d113d6-cd8f-4b3a-86b7-b529372a2f8d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.988728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d113d6-cd8f-4b3a-86b7-b529372a2f8d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.988728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d113d6-cd8f-4b3a-86b7-b529372a2f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.009077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-58d113d6-cd8f-4b3a-86b7-b529372a2f8d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:47.010287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-58d113d6-cd8f-4b3a-86b7-b529372a2f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:47.018051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1105/2181] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:47.022256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b050-ad74-49a4-8268-a5b59605858d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.032463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b050-ad74-49a4-8268-a5b59605858d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.037699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.038307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.038581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.038850 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.039581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:47.039815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.041414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.054957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1077/2182] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:47.059459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d113d6-cd8f-4b3a-86b7-b529372a2f8d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.068724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d113d6-cd8f-4b3a-86b7-b529372a2f8d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.073314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.073907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.074186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.074455 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.075076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.077004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.103583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1106/2183] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:47 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:47.109045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.115267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:47.122006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4189c-ad02-4c4e-b2ae-07056458e270 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.182217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.182966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.183246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.183538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.188766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.188766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.207695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1078/2184] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:47 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:47.211847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.222082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b5dd9-322c-4f71-86e8-d4b1b0931e92 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.227298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.227911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.228190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.228575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.229218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.230862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.241191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1107/2185] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:47.245466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.257133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1467d4-1147-4464-90e4-06a7a1cc5a23 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.262133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de09ee-779d-46db-b735-d39e302a4e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.262773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de09ee-779d-46db-b735-d39e302a4e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.263048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de09ee-779d-46db-b735-d39e302a4e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.263316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de09ee-779d-46db-b735-d39e302a4e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.264046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de09ee-779d-46db-b735-d39e302a4e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.265651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de09ee-779d-46db-b735-d39e302a4e99 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.276612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1079/2186] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:47.280672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.290965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea5b69-478a-4cc1-b319-810ef9d61714 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.295572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.296211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.296488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.296755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] PATH_INFO: `/v3/projects/3634ebcfe3704664b2c10e4aefe5c19e/users/c91108d957624e71a70cf9d9a46bf024/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.311685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3634ebcfe3704664b2c10e4aefe5c19e, user_id=c91108d957624e71a70cf9d9a46bf024, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.314312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.326744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1108/2187] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:47 2022] PUT /identity/v3/projects/3634ebcfe3704664b2c10e4aefe5c19e/users/c91108d957624e71a70cf9d9a46bf024/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:47.337840 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a778e-5968-4e9d-80e6-50853fe6ebf6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.338227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a778e-5968-4e9d-80e6-50853fe6ebf6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.338577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a778e-5968-4e9d-80e6-50853fe6ebf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.375168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2b5a778e-5968-4e9d-80e6-50853fe6ebf6 None None] MFA Rules not processed for user `c91108d957624e71a70cf9d9a46bf024`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:47.376482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2b5a778e-5968-4e9d-80e6-50853fe6ebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:47.453526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1080/2188] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:47.465855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.467654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:47.527373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc853-93dc-425b-aa93-f12ee62273d5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.531227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881ed2b-b06a-4b71-b053-8ee307801a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.531921 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881ed2b-b06a-4b71-b053-8ee307801a8e service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.532229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881ed2b-b06a-4b71-b053-8ee307801a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.532521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881ed2b-b06a-4b71-b053-8ee307801a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.543349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881ed2b-b06a-4b71-b053-8ee307801a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.544853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881ed2b-b06a-4b71-b053-8ee307801a8e service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.554402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1109/2189] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:47.581095 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a778e-5968-4e9d-80e6-50853fe6ebf6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.665998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a778e-5968-4e9d-80e6-50853fe6ebf6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.671815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.672612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.673067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.673508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] PATH_INFO: `/v3/users/c91108d957624e71a70cf9d9a46bf024` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.677986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91108d957624e71a70cf9d9a46bf024)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.677986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.696231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:47.751312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91108d957624e71a70cf9d9a46bf024'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:47.752461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91108d957624e71a70cf9d9a46bf024'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:47.766385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91108d957624e71a70cf9d9a46bf024'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:47.774951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91108d957624e71a70cf9d9a46bf024'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:47.775707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91108d957624e71a70cf9d9a46bf024'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:47.797126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1081/2190] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:47 2022] DELETE /identity/v3/users/c91108d957624e71a70cf9d9a46bf024 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:47.801906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.802186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.802470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.828880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:47.831446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:47.899586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1110/2191] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:47.909672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.911185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:47.973035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a2dcb-1b79-478d-9464-0b1f78f00e27 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.976599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc0733-5173-46d9-9c25-4ce415f2a592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.977201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc0733-5173-46d9-9c25-4ce415f2a592 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.977477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc0733-5173-46d9-9c25-4ce415f2a592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.977744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc0733-5173-46d9-9c25-4ce415f2a592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.986406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc0733-5173-46d9-9c25-4ce415f2a592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.987851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc0733-5173-46d9-9c25-4ce415f2a592 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.997055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1082/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:48.279272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.280868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:48.294109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9129e8-0bef-454f-a169-a378baa290cc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.299526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b448db3-9134-4674-a963-e684981014be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.299763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448db3-9134-4674-a963-e684981014be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.304440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448db3-9134-4674-a963-e684981014be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.304440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448db3-9134-4674-a963-e684981014be admin admin] PATH_INFO: `/v3/projects/3634ebcfe3704664b2c10e4aefe5c19e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.304440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b448db3-9134-4674-a963-e684981014be admin admin] RBAC: Authorizing `identity:delete_project(project_id=3634ebcfe3704664b2c10e4aefe5c19e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.305030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b448db3-9134-4674-a963-e684981014be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.318438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:48.357935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b448db3-9134-4674-a963-e684981014be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3634ebcfe3704664b2c10e4aefe5c19e', 'request_id': 'req-2b448db3-9134-4674-a963-e684981014be'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:48.364931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b448db3-9134-4674-a963-e684981014be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3634ebcfe3704664b2c10e4aefe5c19e', 'request_id': 'req-2b448db3-9134-4674-a963-e684981014be'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:48.370511 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b448db3-9134-4674-a963-e684981014be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3634ebcfe3704664b2c10e4aefe5c19e', 'request_id': 'req-2b448db3-9134-4674-a963-e684981014be'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:48.383977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b448db3-9134-4674-a963-e684981014be admin admin] The token cache is being invalidate because project 3634ebcfe3704664b2c10e4aefe5c19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:48.384271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b448db3-9134-4674-a963-e684981014be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:48.384918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b448db3-9134-4674-a963-e684981014be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:48.386390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1111/2193] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:48 2022] DELETE /identity/v3/projects/3634ebcfe3704664b2c10e4aefe5c19e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:48.392116 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331699c-f717-4208-bf40-30377ddc4b55 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.392410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331699c-f717-4208-bf40-30377ddc4b55 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.392690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331699c-f717-4208-bf40-30377ddc4b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.413118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1331699c-f717-4208-bf40-30377ddc4b55 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:48.414333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1331699c-f717-4208-bf40-30377ddc4b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:48.485247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1083/2194] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:48.494530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341c90-19be-45fd-b7fd-1d7cbba55ca2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.494844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341c90-19be-45fd-b7fd-1d7cbba55ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.495127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341c90-19be-45fd-b7fd-1d7cbba55ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.516003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0e341c90-19be-45fd-b7fd-1d7cbba55ca2 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:48.517212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e341c90-19be-45fd-b7fd-1d7cbba55ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:48.525359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1112/2195] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:48.529530 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331699c-f717-4208-bf40-30377ddc4b55 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.540122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331699c-f717-4208-bf40-30377ddc4b55 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.544758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.545364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.545645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.545950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.546593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:48.546898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.548528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.560890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1084/2196] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:48.565196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341c90-19be-45fd-b7fd-1d7cbba55ca2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.574344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341c90-19be-45fd-b7fd-1d7cbba55ca2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.579184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.579834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.580115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.580385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.581323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.583273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.611314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1113/2197] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:48 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:48.616145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.625849 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc359385-97c5-478d-befd-83e4c5396fac admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.647606 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:48.698151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.698756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.699036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.699307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.699988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.701896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.731024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1085/2198] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:48 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:48.732582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.743807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b415f-35f0-4c45-807a-b91410920197 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.748857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.749492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.749769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.750039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.750688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.752323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.762867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1114/2199] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:48.766768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.777399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d6d4b-3b2b-40d9-9aae-7c1ac7f8059a admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.782105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca8a5-b013-4b12-96fc-afa683d4663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.782736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca8a5-b013-4b12-96fc-afa683d4663f admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.783019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca8a5-b013-4b12-96fc-afa683d4663f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.783292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca8a5-b013-4b12-96fc-afa683d4663f admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.783967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bca8a5-b013-4b12-96fc-afa683d4663f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.785572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bca8a5-b013-4b12-96fc-afa683d4663f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.796930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1086/2200] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:03:48.801116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.812142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa8a79-e4cb-4a90-ae1e-f212af110dfc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.816912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.817648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.817995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.818273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] PATH_INFO: `/v3/projects/82213ebf52f24af49758dd17e5e1d129/users/3cbefab6dddb45c381158126d203f66b/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.832609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82213ebf52f24af49758dd17e5e1d129, user_id=3cbefab6dddb45c381158126d203f66b, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.835359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.851659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1115/2201] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:48 2022] PUT /identity/v3/projects/82213ebf52f24af49758dd17e5e1d129/users/3cbefab6dddb45c381158126d203f66b/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:48.856757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252adf30-7168-42e6-ad48-17694a82ec8e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.857057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252adf30-7168-42e6-ad48-17694a82ec8e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.857330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252adf30-7168-42e6-ad48-17694a82ec8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.878755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-252adf30-7168-42e6-ad48-17694a82ec8e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:48.880651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-252adf30-7168-42e6-ad48-17694a82ec8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:48.950074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1087/2202] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:03:48.959732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.972597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:49.055368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59662785-828d-4b27-ac3e-bad33f3d7271 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:49.059026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:49.059963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:49.060244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:49.060520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:49.070505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:49.072066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:49.082930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1116/2203] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:53.291407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7520-35e3-4840-ac0d-993eee296597 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.292331 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7520-35e3-4840-ac0d-993eee296597 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.292763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7520-35e3-4840-ac0d-993eee296597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.325433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ba9c7520-35e3-4840-ac0d-993eee296597 None None] MFA Rules not processed for user `3cbefab6dddb45c381158126d203f66b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:53.330791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ba9c7520-35e3-4840-ac0d-993eee296597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:53.430672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1088/2204] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:53.436128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.452079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac51451-e032-48af-8071-fe8b7a7344f5 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.462586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.463282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.463484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.463778 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.465764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.467824 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.514431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1117/2205] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:53 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:53.516196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7520-35e3-4840-ac0d-993eee296597 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.529322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7520-35e3-4840-ac0d-993eee296597 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.568419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:53.603887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.604501 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.604779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.605047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.605683 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.607619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.630527 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1089/2206] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:53 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:53.639195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.663379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34531ba6-4267-4be3-b5df-e05922b54a91 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.671049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.671812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.672096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.672386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.673028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.674835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.687591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1118/2207] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:53.693169 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.709888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5278-f333-49bb-93e5-41f9f1a7a7e4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.722601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.724369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.729329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.729793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] PATH_INFO: `/v3/projects/bddf9b8f9d454fd789a20e927a5692f9/users/96db37066a8b4e3ba2fed7caaee50398/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.751403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddf9b8f9d454fd789a20e927a5692f9, user_id=96db37066a8b4e3ba2fed7caaee50398, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.754254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.775206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1090/2208] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:53 2022] PUT /identity/v3/projects/bddf9b8f9d454fd789a20e927a5692f9/users/96db37066a8b4e3ba2fed7caaee50398/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:53.779054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.791966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9f1f6-9042-4f32-a4cd-37a236bb044b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.804023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:53.885819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.886544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.886832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.887174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.888008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.889699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.902245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1119/2209] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:53.906457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.917583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aeab0-61ef-43d8-a6db-6ad9d943a37d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.922838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d8000-613e-42ef-bbfa-868c1e4fb97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.923639 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d8000-613e-42ef-bbfa-868c1e4fb97a admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.924000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d8000-613e-42ef-bbfa-868c1e4fb97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.924560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d8000-613e-42ef-bbfa-868c1e4fb97a admin admin] PATH_INFO: `/v3/projects/bddf9b8f9d454fd789a20e927a5692f9/users/96db37066a8b4e3ba2fed7caaee50398/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.926776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d8000-613e-42ef-bbfa-868c1e4fb97a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddf9b8f9d454fd789a20e927a5692f9, user_id=96db37066a8b4e3ba2fed7caaee50398, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.929558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d8000-613e-42ef-bbfa-868c1e4fb97a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.954733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1091/2210] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:53 2022] PUT /identity/v3/projects/bddf9b8f9d454fd789a20e927a5692f9/users/96db37066a8b4e3ba2fed7caaee50398/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:54.726431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:54.729378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:54.745491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5b98e-2dbd-4679-b0ac-4fecaa536b0c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:54.887504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:54.888392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:54.888684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:54.889090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] PATH_INFO: `/v3/users/1926f8ca5b9e4cb1a721a71510e52404` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:54.902968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1926f8ca5b9e4cb1a721a71510e52404)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:54.906838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:54.974140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1926f8ca5b9e4cb1a721a71510e52404'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:54.981970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1926f8ca5b9e4cb1a721a71510e52404'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:55.005540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1926f8ca5b9e4cb1a721a71510e52404'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:55.006380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1926f8ca5b9e4cb1a721a71510e52404'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:55.007611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1926f8ca5b9e4cb1a721a71510e52404'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:55.039287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1120/2211] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:03:54 2022] DELETE /identity/v3/users/1926f8ca5b9e4cb1a721a71510e52404 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:55.043895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.044431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.044431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.071936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:55.074111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:55.179829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1092/2212] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:55.210816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.212310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:55.249284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:55.293383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793ce8c-4f14-467e-80cc-332c3911dc6b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.298582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc329c9-8756-4a88-91f7-822703f1488a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.299475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc329c9-8756-4a88-91f7-822703f1488a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.300058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc329c9-8756-4a88-91f7-822703f1488a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.300259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc329c9-8756-4a88-91f7-822703f1488a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.314328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc329c9-8756-4a88-91f7-822703f1488a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.316495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc329c9-8756-4a88-91f7-822703f1488a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:55.337543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1121/2213] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:55.867547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.873548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:55.912321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f3a86-66c1-4fa1-8783-2f51053d6d99 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.927040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.927782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.928057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.928378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] PATH_INFO: `/v3/projects/abb14f47b1654c99ae6989f276bc629c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.929921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb14f47b1654c99ae6989f276bc629c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.938044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.028093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb14f47b1654c99ae6989f276bc629c', 'request_id': 'req-67b699b4-495f-43b4-a9dc-365996b7772b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:56.035437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb14f47b1654c99ae6989f276bc629c', 'request_id': 'req-67b699b4-495f-43b4-a9dc-365996b7772b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:56.053886 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb14f47b1654c99ae6989f276bc629c', 'request_id': 'req-67b699b4-495f-43b4-a9dc-365996b7772b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:56.068681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] The token cache is being invalidate because project abb14f47b1654c99ae6989f276bc629c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:03:56.069146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:56.072306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-67b699b4-495f-43b4-a9dc-365996b7772b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:03:56.073916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1093/2214] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:03:55 2022] DELETE /identity/v3/projects/abb14f47b1654c99ae6989f276bc629c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:56.079087 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de36aa-02da-4d13-aec5-5e0b0eef2632 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.079746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de36aa-02da-4d13-aec5-5e0b0eef2632 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.080273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de36aa-02da-4d13-aec5-5e0b0eef2632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.135912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46de36aa-02da-4d13-aec5-5e0b0eef2632 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:56.138998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46de36aa-02da-4d13-aec5-5e0b0eef2632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:56.260564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1122/2215] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:56.269429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5841fe-6573-4b83-b962-2ff650469825 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.269746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5841fe-6573-4b83-b962-2ff650469825 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.270094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5841fe-6573-4b83-b962-2ff650469825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.289699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:56.308602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bf5841fe-6573-4b83-b962-2ff650469825 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:56.310699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bf5841fe-6573-4b83-b962-2ff650469825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:56.326350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1094/2216] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:56.327144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de36aa-02da-4d13-aec5-5e0b0eef2632 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.339943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de36aa-02da-4d13-aec5-5e0b0eef2632 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.347771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.348701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.349144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.349502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.350176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:56.350748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.352582 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.366046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1123/2217] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:56.371926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5841fe-6573-4b83-b962-2ff650469825 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.383276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5841fe-6573-4b83-b962-2ff650469825 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.389015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.390079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.390565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.391029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.392046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.394934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.471096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1095/2218] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:56 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:56.475309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.490609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169559bf-d6ab-4bf8-bdf7-77534b06e0fb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.594700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.595384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.595702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.595981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.596646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.599384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.628527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1124/2219] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:56 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:56.633511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.644831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864383d2-1b4d-496b-ae65-abf995dcb909 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.649952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.650593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.651048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.651323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.652011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.654031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.673980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1096/2220] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:56.677446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.688553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0c752-20c2-4c3b-b098-269cef0e8cae admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.694568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528a929-e1e5-4430-ac93-c0ce47b5cc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.695213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528a929-e1e5-4430-ac93-c0ce47b5cc74 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.695539 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528a929-e1e5-4430-ac93-c0ce47b5cc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.696097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528a929-e1e5-4430-ac93-c0ce47b5cc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.696794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528a929-e1e5-4430-ac93-c0ce47b5cc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.698564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528a929-e1e5-4430-ac93-c0ce47b5cc74 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.719568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1125/2221] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:56.723045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.741740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1d21f-ede6-4eed-92af-db28f9172aa0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.749613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.750438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.750818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.751213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] PATH_INFO: `/v3/projects/6cef443be24742f1a8e2fafd7ded92a0/users/afc3b46290cf4cf0ab6c52b17e30b0f4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.770463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cef443be24742f1a8e2fafd7ded92a0, user_id=afc3b46290cf4cf0ab6c52b17e30b0f4, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.774036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.807987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1097/2222] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:56 2022] PUT /identity/v3/projects/6cef443be24742f1a8e2fafd7ded92a0/users/afc3b46290cf4cf0ab6c52b17e30b0f4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:56.816155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a56e0-9415-47cc-b28f-14db1f035f3b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.816435 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a56e0-9415-47cc-b28f-14db1f035f3b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.816775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a56e0-9415-47cc-b28f-14db1f035f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.872624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-448a56e0-9415-47cc-b28f-14db1f035f3b None None] MFA Rules not processed for user `afc3b46290cf4cf0ab6c52b17e30b0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:56.875822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-448a56e0-9415-47cc-b28f-14db1f035f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:56.908775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:57.012542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:57.014374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1126/2223] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:57.121309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f363e-576b-4f00-99e1-12879571e0ac admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:57.133050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:57.133387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.133735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.133911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.134652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:57.138084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:57.221105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1098/2224] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:03:57 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:57.225867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a56e0-9415-47cc-b28f-14db1f035f3b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:57.260198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a56e0-9415-47cc-b28f-14db1f035f3b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:57.372210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:57.373050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.373479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.373945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.374734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:57.376991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:57.412813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1127/2225] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:57 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:57.417089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:57.431914 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be93c6-ba57-45fc-b09a-7d626aeac698 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:57.437563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:57.438600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.439112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.439846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.440799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:57.443224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:57.458934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1099/2226] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:57.462931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:57.476112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013eed1-8994-4fab-ba63-8642a676238f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:57.491220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:57.492619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.492989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.493333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] PATH_INFO: `/v3/projects/8f9c183932324793915528d2dd7ea918/users/778ed95bf18e4c07ba57c7e9fbff238c/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.514411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9c183932324793915528d2dd7ea918, user_id=778ed95bf18e4c07ba57c7e9fbff238c, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:57.518167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:57.544817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1128/2227] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:57 2022] PUT /identity/v3/projects/8f9c183932324793915528d2dd7ea918/users/778ed95bf18e4c07ba57c7e9fbff238c/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:57.549183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:57.564815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2948c-00e3-41ac-b618-70ed20ffc400 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:57.646622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeed31a-5c42-4086-a984-3fb167a4dce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:57.647607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed31a-5c42-4086-a984-3fb167a4dce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.648143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed31a-5c42-4086-a984-3fb167a4dce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.648621 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed31a-5c42-4086-a984-3fb167a4dce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.649558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeed31a-5c42-4086-a984-3fb167a4dce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:57.651997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeed31a-5c42-4086-a984-3fb167a4dce5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:57.668239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:57.669311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1100/2228] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:57.682064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957730f-b9f4-45d8-b849-f4fa4aefc55d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:57.688992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:57.690091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.690541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.691447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] PATH_INFO: `/v3/projects/8f9c183932324793915528d2dd7ea918/users/778ed95bf18e4c07ba57c7e9fbff238c/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.701258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9c183932324793915528d2dd7ea918, user_id=778ed95bf18e4c07ba57c7e9fbff238c, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:57.717681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:57.753045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898430-b614-418e-ace2-f5d0b17a0a3a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.753918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898430-b614-418e-ace2-f5d0b17a0a3a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.754657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898430-b614-418e-ace2-f5d0b17a0a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.762534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1129/2229] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:03:57 2022] PUT /identity/v3/projects/8f9c183932324793915528d2dd7ea918/users/778ed95bf18e4c07ba57c7e9fbff238c/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:57.797309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:57.811025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-04898430-b614-418e-ace2-f5d0b17a0a3a None None] MFA Rules not processed for user `778ed95bf18e4c07ba57c7e9fbff238c`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:57.815097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-04898430-b614-418e-ace2-f5d0b17a0a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:58.032842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1101/2230] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:03:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:03:58.046095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:58.048032 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:58.174707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0df18-5099-4a6f-ace0-b6fd8d60960c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:58.186016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e252e8-3d53-4091-87a2-94401162e96f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:58.186653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e252e8-3d53-4091-87a2-94401162e96f service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.186933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e252e8-3d53-4091-87a2-94401162e96f service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.187206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e252e8-3d53-4091-87a2-94401162e96f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.203323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e252e8-3d53-4091-87a2-94401162e96f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:58.205158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e252e8-3d53-4091-87a2-94401162e96f service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:58.218938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1130/2231] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:03:58.331254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898430-b614-418e-ace2-f5d0b17a0a3a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:58.347644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898430-b614-418e-ace2-f5d0b17a0a3a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:58.361226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:58.362240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.362535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.362890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.468372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:58.469833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:58.486416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1102/2232] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:03:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:00.028549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:00.029083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:00.029499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:00.068502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] MFA Rules not processed for user `96db37066a8b4e3ba2fed7caaee50398`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:00.069725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:00.075893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:00.157669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1131/2233] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:00.172467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:00.173926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:00.340824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c938-7652-4f99-ab38-6f71492f0a8f service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:00.364836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32414d-ffa4-4b06-9282-3a5f191cf40f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:00.365863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32414d-ffa4-4b06-9282-3a5f191cf40f service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:00.366245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32414d-ffa4-4b06-9282-3a5f191cf40f service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:00.366631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32414d-ffa4-4b06-9282-3a5f191cf40f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:00.372844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dc32414d-ffa4-4b06-9282-3a5f191cf40f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:00.441694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:00.481854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32414d-ffa4-4b06-9282-3a5f191cf40f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:00.483465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32414d-ffa4-4b06-9282-3a5f191cf40f service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:00.497052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1103/2234] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:04:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:10.154022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=27 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:10.367733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:10.369236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:10.449532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc35e54-2806-4ce4-ba35-9dd1c4342582 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:10.455120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:10.455812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:10.456283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:10.456454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] PATH_INFO: `/v3/users/62802bf56af64d6cbe51c8179b311b87` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:10.457707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=62802bf56af64d6cbe51c8179b311b87)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:10.459706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:10.491051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=13 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:10.522929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62802bf56af64d6cbe51c8179b311b87'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:10.533673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62802bf56af64d6cbe51c8179b311b87'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:10.546992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62802bf56af64d6cbe51c8179b311b87'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:10.547784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62802bf56af64d6cbe51c8179b311b87'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:10.548401 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62802bf56af64d6cbe51c8179b311b87'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:10.562562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1132/2235] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:04:10 2022] DELETE /identity/v3/users/62802bf56af64d6cbe51c8179b311b87 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:10.567193 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:10.567481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:10.567792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:10.586544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:10.587798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:10.646729 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1104/2236] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:10.656416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:10.657852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:10.718394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd9fe-a67b-40fa-a2a0-5b0d105560bf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:10.722120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:10.722721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:10.723636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:10.723944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:10.733222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:10.734685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:10.744360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1133/2237] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:11.118007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:11.119454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:11.137910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30fcfd-d564-4e2a-92c2-6aa70de06e6b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:11.142939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:11.143637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:11.144005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:11.144374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] PATH_INFO: `/v3/projects/e4cbb840c67c4bbaa6bad03eecdb7cdc` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:11.145625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4cbb840c67c4bbaa6bad03eecdb7cdc)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:11.147884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:11.196012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4cbb840c67c4bbaa6bad03eecdb7cdc', 'request_id': 'req-793316b1-3b05-43ef-b40e-c0d7d0578f67'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:11.202143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4cbb840c67c4bbaa6bad03eecdb7cdc', 'request_id': 'req-793316b1-3b05-43ef-b40e-c0d7d0578f67'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:11.209950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4cbb840c67c4bbaa6bad03eecdb7cdc', 'request_id': 'req-793316b1-3b05-43ef-b40e-c0d7d0578f67'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:11.220277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] The token cache is being invalidate because project e4cbb840c67c4bbaa6bad03eecdb7cdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:04:11.220564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:11.221269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-793316b1-3b05-43ef-b40e-c0d7d0578f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:11.223326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1105/2238] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:04:11 2022] DELETE /identity/v3/projects/e4cbb840c67c4bbaa6bad03eecdb7cdc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:20.740121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:21.217118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:29.475512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:29.477303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:29.490475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b63f9-ae5d-4300-bd9f-a33023053a20 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:29.549763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:29.550429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:29.550707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:29.550978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] PATH_INFO: `/v3/users/081ead72315e4b58bea1785367c0c6f5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:29.552385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081ead72315e4b58bea1785367c0c6f5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:29.554326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:29.601823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081ead72315e4b58bea1785367c0c6f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:29.609556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081ead72315e4b58bea1785367c0c6f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:29.624158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081ead72315e4b58bea1785367c0c6f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:29.625033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081ead72315e4b58bea1785367c0c6f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:29.625723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081ead72315e4b58bea1785367c0c6f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:29.640687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1134/2239] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:04:29 2022] DELETE /identity/v3/users/081ead72315e4b58bea1785367c0c6f5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:29.645244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:29.645735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:29.645936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:29.665954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:29.667413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:29.735628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1106/2240] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:29.751957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:29.751957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:29.822659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7955-f536-4355-9b75-015b94d5f202 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:29.829909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a719b8-2005-4c49-8c13-e48c2384e838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:29.831211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a719b8-2005-4c49-8c13-e48c2384e838 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:29.832031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a719b8-2005-4c49-8c13-e48c2384e838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:29.832436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a719b8-2005-4c49-8c13-e48c2384e838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:29.844516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a719b8-2005-4c49-8c13-e48c2384e838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:29.846556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a719b8-2005-4c49-8c13-e48c2384e838 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:29.863995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1135/2241] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:30.250902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.252332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:30.264004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a2d78-fd4a-4f44-aa24-18eb4899e071 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.269462 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.270096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.270376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.270646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] PATH_INFO: `/v3/projects/ffa05efb090b4e9199ce821347d00e57` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.271991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa05efb090b4e9199ce821347d00e57)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.274025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:30.322842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa05efb090b4e9199ce821347d00e57', 'request_id': 'req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:30.329261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa05efb090b4e9199ce821347d00e57', 'request_id': 'req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:30.337206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa05efb090b4e9199ce821347d00e57', 'request_id': 'req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:30.353793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] The token cache is being invalidate because project ffa05efb090b4e9199ce821347d00e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:04:30.354270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:30.355146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-bf51dc5f-ac21-456c-9e5d-03e5efcd4e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:30.357356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1107/2242] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:04:30 2022] DELETE /identity/v3/projects/ffa05efb090b4e9199ce821347d00e57 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:30.364774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8285b-f924-4ae2-a097-850a021acf16 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.365066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8285b-f924-4ae2-a097-850a021acf16 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.365373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8285b-f924-4ae2-a097-850a021acf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.389676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-25f8285b-f924-4ae2-a097-850a021acf16 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:30.390961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25f8285b-f924-4ae2-a097-850a021acf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:30.408838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:30.456218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1136/2243] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:30.464403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea902-5324-4797-94f5-4617bf93178b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.464692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea902-5324-4797-94f5-4617bf93178b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.464967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea902-5324-4797-94f5-4617bf93178b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.484776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1b4ea902-5324-4797-94f5-4617bf93178b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:30.485998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1b4ea902-5324-4797-94f5-4617bf93178b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:30.493894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1108/2244] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:30.499756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8285b-f924-4ae2-a097-850a021acf16 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.510173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8285b-f924-4ae2-a097-850a021acf16 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.514890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.515617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.515977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.516326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.517017 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:30.517379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.519095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:30.534768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1137/2245] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:30.536246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ea902-5324-4797-94f5-4617bf93178b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.545449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ea902-5324-4797-94f5-4617bf93178b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.550308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.550907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.551184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.551452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.552368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.554295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:30.585526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1109/2246] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:04:30 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:04:30.590336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.605012 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c55af-18ad-4041-9e4b-d3f9c8a7d18e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.660556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.661163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.661442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.661711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.662334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.664300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:30.685482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1138/2247] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:04:30 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:30.689542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.700516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c396175-813e-482c-a7f5-8d716d8ad162 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.705174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.705774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.706059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.706338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.707008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.708935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:30.720419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1110/2248] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:04:30.724391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.734327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b6626-328b-4475-9d7b-57fda94dd92f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.738892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46d0d6-90cd-449b-8b40-7bc58fffd7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.739479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46d0d6-90cd-449b-8b40-7bc58fffd7fe admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.739782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46d0d6-90cd-449b-8b40-7bc58fffd7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.740050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46d0d6-90cd-449b-8b40-7bc58fffd7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.740678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46d0d6-90cd-449b-8b40-7bc58fffd7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.742293 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46d0d6-90cd-449b-8b40-7bc58fffd7fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:30.752571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1139/2249] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:04:30.756709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.767001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40693685-d794-4f95-b837-43012f2fc614 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.771752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.772572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.772957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.773237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] PATH_INFO: `/v3/projects/82379127940c44ac99bab9fcb8bb80d2/users/3509ad99de7e4604abd09911172f8c65/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.788026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82379127940c44ac99bab9fcb8bb80d2, user_id=3509ad99de7e4604abd09911172f8c65, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.790660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:30.803243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1111/2250] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:04:30 2022] PUT /identity/v3/projects/82379127940c44ac99bab9fcb8bb80d2/users/3509ad99de7e4604abd09911172f8c65/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:30.814852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e3a61-01af-480d-b5f4-988072a575ef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.815132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e3a61-01af-480d-b5f4-988072a575ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.815399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e3a61-01af-480d-b5f4-988072a575ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.843729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f80e3a61-01af-480d-b5f4-988072a575ef None None] MFA Rules not processed for user `3509ad99de7e4604abd09911172f8c65`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:30.845035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f80e3a61-01af-480d-b5f4-988072a575ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:30.910612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1140/2251] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:30.922141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:30.923913 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:30.946975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:30.982384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f05108-4704-40ea-84b6-6531e49384b9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:30.986028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57400679-dd2f-4da8-8a2c-61a526ea86fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:30.986623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57400679-dd2f-4da8-8a2c-61a526ea86fb service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:30.986903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57400679-dd2f-4da8-8a2c-61a526ea86fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:30.987191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57400679-dd2f-4da8-8a2c-61a526ea86fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:30.995954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57400679-dd2f-4da8-8a2c-61a526ea86fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:30.997386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57400679-dd2f-4da8-8a2c-61a526ea86fb service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:31.006252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1112/2252] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:04:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:37.942450 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e3a61-01af-480d-b5f4-988072a575ef None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:37.961409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:38.007358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e3a61-01af-480d-b5f4-988072a575ef None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.012001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.012644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.012925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.013197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] PATH_INFO: `/v3/users/3509ad99de7e4604abd09911172f8c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.014324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3509ad99de7e4604abd09911172f8c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:38.016301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:38.063838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3509ad99de7e4604abd09911172f8c65'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.070545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3509ad99de7e4604abd09911172f8c65'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.082912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3509ad99de7e4604abd09911172f8c65'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.083604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3509ad99de7e4604abd09911172f8c65'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.084208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3509ad99de7e4604abd09911172f8c65'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.097011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1141/2253] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:04:37 2022] DELETE /identity/v3/users/3509ad99de7e4604abd09911172f8c65 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:38.101714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.102786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.102892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.124164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:38.125408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:38.185346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1113/2254] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:04:38.195346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:38.196840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:38.260488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cee56-0179-491a-bf8f-0e9438e91a06 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.263888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a082dca-f234-43de-9ce3-92655f058f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.264635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a082dca-f234-43de-9ce3-92655f058f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.264946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a082dca-f234-43de-9ce3-92655f058f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.265258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a082dca-f234-43de-9ce3-92655f058f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.280617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a082dca-f234-43de-9ce3-92655f058f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:38.282138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a082dca-f234-43de-9ce3-92655f058f22 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:38.300447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1142/2255] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:38.619962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:38.621762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:38.640916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723ca6e-1be4-44ad-920e-5f773066da67 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.646863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.647523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.647815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.648086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] PATH_INFO: `/v3/projects/82379127940c44ac99bab9fcb8bb80d2` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.649838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82379127940c44ac99bab9fcb8bb80d2)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:38.651770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:38.705809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82379127940c44ac99bab9fcb8bb80d2', 'request_id': 'req-858ea948-c97f-4a58-b9d3-ec8f8d729679'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.712423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82379127940c44ac99bab9fcb8bb80d2', 'request_id': 'req-858ea948-c97f-4a58-b9d3-ec8f8d729679'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.719972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82379127940c44ac99bab9fcb8bb80d2', 'request_id': 'req-858ea948-c97f-4a58-b9d3-ec8f8d729679'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.730286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] The token cache is being invalidate because project 82379127940c44ac99bab9fcb8bb80d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:04:38.730570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.731188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:38.732686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1114/2256] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:04:38 2022] DELETE /identity/v3/projects/82379127940c44ac99bab9fcb8bb80d2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:47.811587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:47.812076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:47.812165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:47.833394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:47.834911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:47.880668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:47.910155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1143/2257] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:04:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:47.960162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:47.960981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:48.012863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:48.036227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ea948-c97f-4a58-b9d3-ec8f8d729679 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:48.042863 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436233e9-cbb9-4614-8a61-e222f2a37f9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:48.043478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436233e9-cbb9-4614-8a61-e222f2a37f9c service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:48.043793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436233e9-cbb9-4614-8a61-e222f2a37f9c service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:48.044067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436233e9-cbb9-4614-8a61-e222f2a37f9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:48.052686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436233e9-cbb9-4614-8a61-e222f2a37f9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:48.054255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436233e9-cbb9-4614-8a61-e222f2a37f9c service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:48.062258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1115/2258] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:04:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:04:49.971671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:49.975595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:50.110062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8991a8-0b38-4713-a600-2ebe59531858 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:50.111473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8991a8-0b38-4713-a600-2ebe59531858 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:50.111694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2433d-9aa4-454b-81ea-bf6b85428ddc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:50.112671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8991a8-0b38-4713-a600-2ebe59531858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:50.125038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:50.125808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:50.126095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:50.127134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] PATH_INFO: `/v3/users/afc3b46290cf4cf0ab6c52b17e30b0f4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:50.129760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc3b46290cf4cf0ab6c52b17e30b0f4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:50.133044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:50.146118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8b8991a8-0b38-4713-a600-2ebe59531858 None None] MFA Rules not processed for user `3cbefab6dddb45c381158126d203f66b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:50.149531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8b8991a8-0b38-4713-a600-2ebe59531858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:50.216031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc3b46290cf4cf0ab6c52b17e30b0f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:50.225578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc3b46290cf4cf0ab6c52b17e30b0f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:50.239663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1116/2259] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:04:50.254718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc3b46290cf4cf0ab6c52b17e30b0f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:50.255560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc3b46290cf4cf0ab6c52b17e30b0f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:50.256314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc3b46290cf4cf0ab6c52b17e30b0f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:50.347168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1144/2260] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:04:49 2022] DELETE /identity/v3/users/afc3b46290cf4cf0ab6c52b17e30b0f4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:50.363456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:50.364952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:50.365712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:50.471279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:50.475866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:50.606386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1117/2261] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:50.616245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:50.618277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:50.706857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b7a6-bbf5-4150-b5e5-fd1895323ab7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:50.706857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:50.706857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:50.706857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:50.707370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:50.723692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:50.725221 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:50.740877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1145/2262] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:51.218117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:51.219719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:51.237776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777a4df-1685-4626-bb0a-cd584ff2ceaa None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:51.255994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:51.257159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:51.257650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:51.258190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] PATH_INFO: `/v3/projects/6cef443be24742f1a8e2fafd7ded92a0` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:51.259769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cef443be24742f1a8e2fafd7ded92a0)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:51.261774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:51.331893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cef443be24742f1a8e2fafd7ded92a0', 'request_id': 'req-3befe14b-f0e8-4316-a315-1a0a16e3ad09'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.341696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cef443be24742f1a8e2fafd7ded92a0', 'request_id': 'req-3befe14b-f0e8-4316-a315-1a0a16e3ad09'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.355090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cef443be24742f1a8e2fafd7ded92a0', 'request_id': 'req-3befe14b-f0e8-4316-a315-1a0a16e3ad09'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.373233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] The token cache is being invalidate because project 6cef443be24742f1a8e2fafd7ded92a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:04:51.374031 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.374903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.377582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1118/2263] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:04:51 2022] DELETE /identity/v3/projects/6cef443be24742f1a8e2fafd7ded92a0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:51.384416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:51.395748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca93f2-1fe5-4129-8344-4128c3095397 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:51.401153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:51.490759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:51.491413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:51.491976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:51.491976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] PATH_INFO: `/v3/users/778ed95bf18e4c07ba57c7e9fbff238c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:51.494399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=778ed95bf18e4c07ba57c7e9fbff238c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:51.497412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:51.612204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '778ed95bf18e4c07ba57c7e9fbff238c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.622451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '778ed95bf18e4c07ba57c7e9fbff238c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.640114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '778ed95bf18e4c07ba57c7e9fbff238c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.640879 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '778ed95bf18e4c07ba57c7e9fbff238c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.641007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-01a360a5-e5c6-4ff3-8b1d-e32a1418a9cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '778ed95bf18e4c07ba57c7e9fbff238c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:51.663522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1146/2264] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:04:51 2022] DELETE /identity/v3/users/778ed95bf18e4c07ba57c7e9fbff238c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:52.101207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:52.103526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:52.112872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:52.121820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befe14b-f0e8-4316-a315-1a0a16e3ad09 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:52.195160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:52.196036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.196313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.196717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] PATH_INFO: `/v3/projects/8f9c183932324793915528d2dd7ea918` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.197880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9c183932324793915528d2dd7ea918)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:52.199960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:52.267792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9c183932324793915528d2dd7ea918', 'request_id': 'req-6bdfce2b-1e13-4a10-b81d-6c29400060ab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:52.280504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9c183932324793915528d2dd7ea918', 'request_id': 'req-6bdfce2b-1e13-4a10-b81d-6c29400060ab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:52.293417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9c183932324793915528d2dd7ea918', 'request_id': 'req-6bdfce2b-1e13-4a10-b81d-6c29400060ab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:52.317168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] The token cache is being invalidate because project 8f9c183932324793915528d2dd7ea918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:04:52.317842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:52.318759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6bdfce2b-1e13-4a10-b81d-6c29400060ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:04:52.321294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1119/2265] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:04:52 2022] DELETE /identity/v3/projects/8f9c183932324793915528d2dd7ea918 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:52.326707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fc77a-b408-4a5f-a62a-7e56d3105dc6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.327271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fc77a-b408-4a5f-a62a-7e56d3105dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.327783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fc77a-b408-4a5f-a62a-7e56d3105dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.381148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5c2fc77a-b408-4a5f-a62a-7e56d3105dc6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:52.383128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c2fc77a-b408-4a5f-a62a-7e56d3105dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:52.485469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1147/2266] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:04:52.494053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dca2d-87e2-453e-bc01-feffb7375096 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.494340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dca2d-87e2-453e-bc01-feffb7375096 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.494607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dca2d-87e2-453e-bc01-feffb7375096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.516520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6c3dca2d-87e2-453e-bc01-feffb7375096 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:52.517760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6c3dca2d-87e2-453e-bc01-feffb7375096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:52.530447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1120/2267] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:52.535105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fc77a-b408-4a5f-a62a-7e56d3105dc6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:52.550165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fc77a-b408-4a5f-a62a-7e56d3105dc6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:52.556883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:52.557668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.557944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.558212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.558906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:52.559208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:52.560835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:52.575223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1148/2268] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:52.578799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dca2d-87e2-453e-bc01-feffb7375096 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:52.591477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dca2d-87e2-453e-bc01-feffb7375096 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:52.600036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:52.600657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.600984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.601382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.602647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:52.604723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:52.643146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1121/2269] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:04:52 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:52.644336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:52.655897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3eaa4-59c9-4ba1-a55d-549b692a1bed admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:52.679335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:52.746698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:52.747607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.747891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.748161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.748812 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:52.750741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:52.775986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1149/2270] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:04:52 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:04:52.781045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:52.793778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14187a-8c61-433c-a84f-df18cc5e0905 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:52.801536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:52.803018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.803307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.803610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.804256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:52.806248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:52.822689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1122/2271] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:04:52.822903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:52.833317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a86d4b-b5cf-411b-8693-9e5f544b7900 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:52.844042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378be9f-548a-46cb-92b0-e4893ff9de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:52.844770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378be9f-548a-46cb-92b0-e4893ff9de2e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.845198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378be9f-548a-46cb-92b0-e4893ff9de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.845673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378be9f-548a-46cb-92b0-e4893ff9de2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.846247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378be9f-548a-46cb-92b0-e4893ff9de2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:52.848068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378be9f-548a-46cb-92b0-e4893ff9de2e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:52.861848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1150/2272] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:04:52.865983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:52.882458 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8207215-5564-4b8a-a2f1-1406fa6d288b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:52.888890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:52.889733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.890089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.890437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] PATH_INFO: `/v3/projects/449e13e49e52413f9bdf141dc393a4be/users/fd479e08b3304683ae84f526e8f76eab/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:52.912950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449e13e49e52413f9bdf141dc393a4be, user_id=fd479e08b3304683ae84f526e8f76eab, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:52.915996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:52.976390 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1123/2273] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:04:52 2022] PUT /identity/v3/projects/449e13e49e52413f9bdf141dc393a4be/users/fd479e08b3304683ae84f526e8f76eab/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:52.978740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ff67-1fe4-4ff7-84fc-91086fa8ee28 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:52.979206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ff67-1fe4-4ff7-84fc-91086fa8ee28 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:52.979713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ff67-1fe4-4ff7-84fc-91086fa8ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.042548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4cd1ff67-1fe4-4ff7-84fc-91086fa8ee28 None None] MFA Rules not processed for user `fd479e08b3304683ae84f526e8f76eab`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:53.044110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4cd1ff67-1fe4-4ff7-84fc-91086fa8ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:53.166522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1151/2274] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:53.171442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.272075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:53.324171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c837651-41fe-4b2f-8c60-fd7225661261 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:53.330452 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:53.331545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.332036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.332531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.333429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:53.339212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:53.383764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1124/2275] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:04:53 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:53.387930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ff67-1fe4-4ff7-84fc-91086fa8ee28 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.401150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ff67-1fe4-4ff7-84fc-91086fa8ee28 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:53.516177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:53.517820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.518321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.518820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.519705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:53.525491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:53.560462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1152/2276] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:04:53 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:04:53.564160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.582339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200e47c-3755-429c-898a-529f7b2d3b19 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:53.595142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:53.596117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.596402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.597120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.600575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:53.602878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:53.621226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1125/2277] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:04:53.625285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.636714 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf68e-0182-4556-8e5b-322d8689d1b6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:53.656626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:53.657622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.657902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.658175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] PATH_INFO: `/v3/projects/392aa7e4e1704fd8b6e01453f861df49/users/57e87622c2f8465ca7969eb17e78afab/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.672270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:53.682645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=392aa7e4e1704fd8b6e01453f861df49, user_id=57e87622c2f8465ca7969eb17e78afab, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:53.685735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:53.702209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1153/2278] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:04:53 2022] PUT /identity/v3/projects/392aa7e4e1704fd8b6e01453f861df49/users/57e87622c2f8465ca7969eb17e78afab/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:04:53.706257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.721175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8bf0e-181c-4a81-a163-e417ea204984 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:53.796252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ee6ec-c915-41b1-bfc3-77dd186bc58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:53.797347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ee6ec-c915-41b1-bfc3-77dd186bc58c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.797787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ee6ec-c915-41b1-bfc3-77dd186bc58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.798208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ee6ec-c915-41b1-bfc3-77dd186bc58c admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.799006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ee6ec-c915-41b1-bfc3-77dd186bc58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:53.800843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ee6ec-c915-41b1-bfc3-77dd186bc58c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:53.817637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1126/2279] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:04:53.822136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.832784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356a865-516e-4a41-99c5-ea7f7c6ffa3c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:53.838028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:53.838698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.838981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.839253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] PATH_INFO: `/v3/projects/392aa7e4e1704fd8b6e01453f861df49/users/57e87622c2f8465ca7969eb17e78afab/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.841270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=392aa7e4e1704fd8b6e01453f861df49, user_id=57e87622c2f8465ca7969eb17e78afab, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:53.844233 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:53.855123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1154/2280] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:04:53 2022] PUT /identity/v3/projects/392aa7e4e1704fd8b6e01453f861df49/users/57e87622c2f8465ca7969eb17e78afab/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:53.865443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe98fd-d56d-42dd-a632-8fc9b043b835 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.866271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe98fd-d56d-42dd-a632-8fc9b043b835 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.866651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe98fd-d56d-42dd-a632-8fc9b043b835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.897704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7abe98fd-d56d-42dd-a632-8fc9b043b835 None None] MFA Rules not processed for user `57e87622c2f8465ca7969eb17e78afab`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:53.899601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7abe98fd-d56d-42dd-a632-8fc9b043b835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:53.985867 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1127/2281] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:04:53.997933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.999707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:54.088047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd331e0-2dc8-4b84-87b4-1d328292d39b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:54.092558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6545ad-07c7-4b05-a8f2-233f03d8f07d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:54.093170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6545ad-07c7-4b05-a8f2-233f03d8f07d service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:54.093584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6545ad-07c7-4b05-a8f2-233f03d8f07d service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:54.093851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6545ad-07c7-4b05-a8f2-233f03d8f07d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:54.178338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6545ad-07c7-4b05-a8f2-233f03d8f07d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:54.180678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6545ad-07c7-4b05-a8f2-233f03d8f07d service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:54.196185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1155/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:04:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:04:59.951101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:59.951659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:59.951895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:59.973208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] MFA Rules not processed for user `e24e6fb4c69d48f7b2972b53fb8ea4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:59.973862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:59.996283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:00.040702 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1128/2283] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:04:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:02.966575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0efe74-9e93-41be-a3b8-e35902420642 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:02.967133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0efe74-9e93-41be-a3b8-e35902420642 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:02.967133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0efe74-9e93-41be-a3b8-e35902420642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:02.986647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ab0efe74-9e93-41be-a3b8-e35902420642 None None] MFA Rules not processed for user `85cd8a90230b4c69886534be67acd943`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:02.987910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab0efe74-9e93-41be-a3b8-e35902420642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:03.042289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1156/2284] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:05:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:04.706061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:04.708117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:04.740064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf39a59-f6ec-4663-a93a-665b9e533731 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:04.745907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920951e9-2eb9-4074-b903-c317414d46b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:04.746777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920951e9-2eb9-4074-b903-c317414d46b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:04.747616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920951e9-2eb9-4074-b903-c317414d46b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:04.747616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920951e9-2eb9-4074-b903-c317414d46b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:04.758166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920951e9-2eb9-4074-b903-c317414d46b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:04.760441 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920951e9-2eb9-4074-b903-c317414d46b9 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:04.777157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1129/2285] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:11.909599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49cde6-8f2c-4768-81bd-d64df9813df6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:11.910131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49cde6-8f2c-4768-81bd-d64df9813df6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:11.910330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49cde6-8f2c-4768-81bd-d64df9813df6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:11.911808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1157/2286] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:05:11.918244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec2d3-4b53-48cc-adc4-a364939419ec None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:11.918715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec2d3-4b53-48cc-adc4-a364939419ec None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:11.918835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec2d3-4b53-48cc-adc4-a364939419ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:11.939815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b7aec2d3-4b53-48cc-adc4-a364939419ec None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:11.943650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b7aec2d3-4b53-48cc-adc4-a364939419ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:12.032929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1130/2287] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:05:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:12.051833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49cde6-8f2c-4768-81bd-d64df9813df6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:12.051833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fa49cde6-8f2c-4768-81bd-d64df9813df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:12.082279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:12.105363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49cde6-8f2c-4768-81bd-d64df9813df6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:12.108889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234782b-4aba-4818-a7a0-3a86b19ec36e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:12.109491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234782b-4aba-4818-a7a0-3a86b19ec36e service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:12.109769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234782b-4aba-4818-a7a0-3a86b19ec36e service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:12.110041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234782b-4aba-4818-a7a0-3a86b19ec36e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:12.119953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234782b-4aba-4818-a7a0-3a86b19ec36e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:12.121379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234782b-4aba-4818-a7a0-3a86b19ec36e service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:12.131995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1158/2288] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:12.220851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92862-ccd1-49f3-9450-9306598a951d None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:12.221058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92862-ccd1-49f3-9450-9306598a951d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:12.222066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92862-ccd1-49f3-9450-9306598a951d None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:12.222788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1131/2289] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:05:12.228447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda52fba-69bc-49d2-9e04-3727d9a77e90 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:12.228754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda52fba-69bc-49d2-9e04-3727d9a77e90 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:12.229040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda52fba-69bc-49d2-9e04-3727d9a77e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:12.253796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eda52fba-69bc-49d2-9e04-3727d9a77e90 None None] MFA Rules not processed for user `c0e10b1ef7454167bd1c606ef4f70bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:12.255223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eda52fba-69bc-49d2-9e04-3727d9a77e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:12.314586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1159/2290] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:05:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:12.326790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef92862-ccd1-49f3-9450-9306598a951d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:12.328594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eef92862-ccd1-49f3-9450-9306598a951d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:12.342127 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef92862-ccd1-49f3-9450-9306598a951d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:12.345718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bbec1-09df-4d60-ae03-fe887059375b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:12.346404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbec1-09df-4d60-ae03-fe887059375b service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:12.346772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbec1-09df-4d60-ae03-fe887059375b service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:12.347088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbec1-09df-4d60-ae03-fe887059375b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:12.354951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bbec1-09df-4d60-ae03-fe887059375b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:12.356403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bbec1-09df-4d60-ae03-fe887059375b service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:12.365063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1132/2291] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:12.482250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41e4cf-e1ea-4f53-81e9-182d09275855 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:12.482544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41e4cf-e1ea-4f53-81e9-182d09275855 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:12.482820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41e4cf-e1ea-4f53-81e9-182d09275855 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:12.484227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1160/2292] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:05:12.489398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:12.489689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:12.489964 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:12.511179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:12.512438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:12.521620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1133/2293] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:05:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:12.532227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41e4cf-e1ea-4f53-81e9-182d09275855 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:12.543809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41e4cf-e1ea-4f53-81e9-182d09275855 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:12.547321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0402a9-558d-4ccb-96b2-c660472bd346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:12.547969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0402a9-558d-4ccb-96b2-c660472bd346 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:12.548252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0402a9-558d-4ccb-96b2-c660472bd346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:12.548522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0402a9-558d-4ccb-96b2-c660472bd346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:12.560388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0402a9-558d-4ccb-96b2-c660472bd346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:12.561819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0402a9-558d-4ccb-96b2-c660472bd346 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:12.573618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1161/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:22.505378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=59 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:22.568239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=40 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:31.693457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:31.694923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:31.706220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4bcc5-83de-46c5-99ea-079c37f37c53 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:31.709768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:31.710365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:31.710644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:31.710924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:31.712724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:31.776732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:31.778147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:31.788575 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1134/2295] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:32.165093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:32.165588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:32.165696 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:32.171120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:32.187759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:32.189688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:32.255121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033f98f-97de-4080-81df-f5b313ef87e6 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:32.260714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:32.261407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:32.262030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:32.262615 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] PATH_INFO: `/v3/users/fd479e08b3304683ae84f526e8f76eab` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:32.263977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd479e08b3304683ae84f526e8f76eab)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:32.267209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:32.281638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1162/2296] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:05:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:32.301641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:32.303359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:32.342354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd479e08b3304683ae84f526e8f76eab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:32.343236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd479e08b3304683ae84f526e8f76eab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:32.351669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd479e08b3304683ae84f526e8f76eab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:32.368768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd479e08b3304683ae84f526e8f76eab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:32.369564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-df21398b-9f65-44f7-9cb1-d205fdd6c916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd479e08b3304683ae84f526e8f76eab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:32.386138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1135/2297] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:05:32 2022] DELETE /identity/v3/users/fd479e08b3304683ae84f526e8f76eab => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:05:32.391528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:32.391904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:32.392321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:32.412557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:32.413913 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:32.423068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:32.437313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4a213-13b7-4b44-be58-b82817542cd2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:32.528268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:32.529674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:32.529958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:32.530232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:32.531071 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1136/2298] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:05:32.534704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:32.547896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:32.549999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:32.609996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:32.631916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7d7e1-2901-4c03-8e5c-7075455795c0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:32.635594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:32.636542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:32.637224 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:32.637424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:32.637543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:32.637838 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:32.653074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1163/2299] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:05:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:05:32.672438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:32.674110 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:32.692055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1137/2300] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:33.343561 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:33.345110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:33.370955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4b95c-2e7e-4f93-b044-4435e6acbcce service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:33.381753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:33.382728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:33.384173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:33.384613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] PATH_INFO: `/v3/projects/449e13e49e52413f9bdf141dc393a4be` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:33.386164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449e13e49e52413f9bdf141dc393a4be)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:33.388350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:33.461123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449e13e49e52413f9bdf141dc393a4be', 'request_id': 'req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.473611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449e13e49e52413f9bdf141dc393a4be', 'request_id': 'req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.481120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449e13e49e52413f9bdf141dc393a4be', 'request_id': 'req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.501252 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] The token cache is being invalidate because project 449e13e49e52413f9bdf141dc393a4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:05:33.501537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.502260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.507295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1164/2301] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:05:33 2022] DELETE /identity/v3/projects/449e13e49e52413f9bdf141dc393a4be => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:05:33.510322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:33.530329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97fc78-9b9c-42b8-a426-34ae17060192 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:33.644414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:33.645266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:33.645706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:33.646528 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] PATH_INFO: `/v3/users/57e87622c2f8465ca7969eb17e78afab` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:33.647761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e87622c2f8465ca7969eb17e78afab)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:33.650150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:33.749242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e87622c2f8465ca7969eb17e78afab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.752904 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e87622c2f8465ca7969eb17e78afab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.768494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e87622c2f8465ca7969eb17e78afab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.771830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:33.795120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e87622c2f8465ca7969eb17e78afab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.796862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-910aefef-f719-42c8-8184-dc104cbc8e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e87622c2f8465ca7969eb17e78afab'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:33.828023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1138/2302] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:05:33 2022] DELETE /identity/v3/users/57e87622c2f8465ca7969eb17e78afab => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:05:35.452564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:35.454019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:35.485522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57dc21-d4e3-4bf1-89e6-4a03aa2e9ff1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:35.640538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:35.641496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:35.641938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:35.642371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] PATH_INFO: `/v3/projects/392aa7e4e1704fd8b6e01453f861df49` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:35.645365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] RBAC: Authorizing `identity:delete_project(project_id=392aa7e4e1704fd8b6e01453f861df49)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:35.647523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:35.768944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392aa7e4e1704fd8b6e01453f861df49', 'request_id': 'req-e6e84b20-052a-40e8-b54f-41d5048b158f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:35.780733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392aa7e4e1704fd8b6e01453f861df49', 'request_id': 'req-e6e84b20-052a-40e8-b54f-41d5048b158f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:35.787613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392aa7e4e1704fd8b6e01453f861df49', 'request_id': 'req-e6e84b20-052a-40e8-b54f-41d5048b158f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:35.836834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] The token cache is being invalidate because project 392aa7e4e1704fd8b6e01453f861df49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:05:35.838101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:35.839567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6e84b20-052a-40e8-b54f-41d5048b158f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:05:35.846104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1165/2303] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:05:35 2022] DELETE /identity/v3/projects/392aa7e4e1704fd8b6e01453f861df49 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:05:35.850812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649942-8838-4f91-b5a6-8fc530e94f44 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:35.852522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649942-8838-4f91-b5a6-8fc530e94f44 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:35.853041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649942-8838-4f91-b5a6-8fc530e94f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:35.906345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-50649942-8838-4f91-b5a6-8fc530e94f44 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:35.907969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-50649942-8838-4f91-b5a6-8fc530e94f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:36.079731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1139/2304] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:36.089161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d842dd-7fae-4b5e-8185-9e22b78fbc35 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.089697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d842dd-7fae-4b5e-8185-9e22b78fbc35 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.090151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d842dd-7fae-4b5e-8185-9e22b78fbc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.101044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:36.143611 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-00d842dd-7fae-4b5e-8185-9e22b78fbc35 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:36.145074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-00d842dd-7fae-4b5e-8185-9e22b78fbc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:36.158449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1166/2305] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:36.162545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50649942-8838-4f91-b5a6-8fc530e94f44 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:36.186133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50649942-8838-4f91-b5a6-8fc530e94f44 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:36.206515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:36.207726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.207726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.207726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.208393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:05:36.208697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:36.210676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:36.260981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d842dd-7fae-4b5e-8185-9e22b78fbc35 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:36.262322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1140/2306] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:36.281979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d842dd-7fae-4b5e-8185-9e22b78fbc35 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:36.296695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:36.297652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.298102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.298583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.299394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:36.306473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:36.384807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1167/2307] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:05:36 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:05:36.389843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:36.403333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92725ae8-393e-42be-a2b3-46f3ce922fb9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:36.550705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:36.553286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.553577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.553846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.554498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:36.556993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:36.603150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1141/2308] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:05:36 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:05:36.607183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:36.623264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752c6e3-73da-4734-b9b7-96d783394c98 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:36.628448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:36.629066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.629429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.629705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.630528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:36.632548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:36.655456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1168/2309] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:05:36.659227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:36.681327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacb88a-44db-4311-90a7-99a8c4cacfab admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:36.696478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f0562-385f-4483-93dd-2619961fb5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:36.697438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f0562-385f-4483-93dd-2619961fb5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.697875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f0562-385f-4483-93dd-2619961fb5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.698301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f0562-385f-4483-93dd-2619961fb5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.699248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f0562-385f-4483-93dd-2619961fb5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:36.701233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f0562-385f-4483-93dd-2619961fb5a4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:36.725433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1142/2310] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:05:36.729251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:36.743657 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c74d2-6df1-4cde-bf86-3703897eb80c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:36.748624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:36.749399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.749681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.749960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] PATH_INFO: `/v3/projects/6c4287a54fa04a60beb7055cf5552370/users/51d77bd769ca4a23808042a742bce663/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.773422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4287a54fa04a60beb7055cf5552370, user_id=51d77bd769ca4a23808042a742bce663, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:36.777431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:36.812665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1169/2311] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:05:36 2022] PUT /identity/v3/projects/6c4287a54fa04a60beb7055cf5552370/users/51d77bd769ca4a23808042a742bce663/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:05:36.824024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9f80-43f8-488e-a138-ef06ba3d422f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:36.824370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9f80-43f8-488e-a138-ef06ba3d422f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:36.824584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9f80-43f8-488e-a138-ef06ba3d422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:36.862497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-145e9f80-43f8-488e-a138-ef06ba3d422f None None] MFA Rules not processed for user `51d77bd769ca4a23808042a742bce663`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:36.864073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-145e9f80-43f8-488e-a138-ef06ba3d422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:36.918644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:37.054272 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1143/2312] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:05:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:37.067882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:37.069765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:37.193877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344ffab-7db2-4a15-a9dd-657c274d3375 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:37.198597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:37.199361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:37.199773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:37.200172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:37.210139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:37.213120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:37.241138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a02461-b6bc-47e6-9943-2df7fcd0c153 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:37.241421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a02461-b6bc-47e6-9943-2df7fcd0c153 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:37.241695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a02461-b6bc-47e6-9943-2df7fcd0c153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:37.293394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1170/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:37.323868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-13a02461-b6bc-47e6-9943-2df7fcd0c153 None None] MFA Rules not processed for user `cf0dc39479d1435f902e10cb6827f5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:37.328107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-13a02461-b6bc-47e6-9943-2df7fcd0c153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:37.485995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1144/2314] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:05:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:43.363202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:43.364664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:43.488133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2567-d2bf-4e00-b906-e642cf7ef3d4 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:43.495269 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:43.495946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:43.496229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:43.496503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:43.499165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:43.516029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:43.518527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:43.531799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1171/2315] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:05:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:05:47.454787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=38 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:53.526695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:57.444880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07494d9-612d-44a4-a875-8c1004c76eea None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:57.445387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07494d9-612d-44a4-a875-8c1004c76eea None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:57.445504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07494d9-612d-44a4-a875-8c1004c76eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:57.464743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b07494d9-612d-44a4-a875-8c1004c76eea None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:57.465926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b07494d9-612d-44a4-a875-8c1004c76eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:57.521225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1145/2316] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 23:05:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:05:57.544625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:57.546046 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:57.619742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655a313-58f3-41eb-9948-d4d3931318a6 service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:57.624973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705ff83-da2c-488f-a5e1-5ce3b212122b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:57.625585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705ff83-da2c-488f-a5e1-5ce3b212122b service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:57.625869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705ff83-da2c-488f-a5e1-5ce3b212122b service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:57.626143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705ff83-da2c-488f-a5e1-5ce3b212122b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:57.634364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705ff83-da2c-488f-a5e1-5ce3b212122b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:57.636107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705ff83-da2c-488f-a5e1-5ce3b212122b service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:57.644169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1172/2317] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:05:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:06:07.513118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=25 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:07.640910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=32 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:22.594703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07494d9-612d-44a4-a875-8c1004c76eea None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:22.658293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07494d9-612d-44a4-a875-8c1004c76eea None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:22.662798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:22.663418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:22.663731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:22.664006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] PATH_INFO: `/v3/users/51d77bd769ca4a23808042a742bce663` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:22.665126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d77bd769ca4a23808042a742bce663)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:22.667050 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:22.712626 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d77bd769ca4a23808042a742bce663'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:22.713551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d77bd769ca4a23808042a742bce663'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:22.721610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d77bd769ca4a23808042a742bce663'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:22.735583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d77bd769ca4a23808042a742bce663'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:22.736295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d77bd769ca4a23808042a742bce663'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:22.749189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1146/2318] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:06:22 2022] DELETE /identity/v3/users/51d77bd769ca4a23808042a742bce663 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:22.753775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:22.754280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:22.754376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:22.773979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:22.775311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:22.840299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1173/2319] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:06:22.851135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:22.852623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:22.909461 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca992b2-d1d1-4f57-af75-2ed56fb4255d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:22.913130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77105ea-89d1-461d-9ec1-2eb9a131ebe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:22.913734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77105ea-89d1-461d-9ec1-2eb9a131ebe7 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:22.914015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77105ea-89d1-461d-9ec1-2eb9a131ebe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:22.914286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77105ea-89d1-461d-9ec1-2eb9a131ebe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:22.923710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77105ea-89d1-461d-9ec1-2eb9a131ebe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:22.925138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77105ea-89d1-461d-9ec1-2eb9a131ebe7 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:22.934822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1147/2320] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:06:23.216802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.218367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:23.235888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302b5b0-28cc-43c5-80d7-474ae0d5feb0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.244006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.244935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.245337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.245749 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] PATH_INFO: `/v3/projects/6c4287a54fa04a60beb7055cf5552370` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.247388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c4287a54fa04a60beb7055cf5552370)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.251201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.303159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c4287a54fa04a60beb7055cf5552370', 'request_id': 'req-7e3b015a-44ef-4ace-b49b-142df9953ed1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:23.309120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c4287a54fa04a60beb7055cf5552370', 'request_id': 'req-7e3b015a-44ef-4ace-b49b-142df9953ed1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:23.315447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c4287a54fa04a60beb7055cf5552370', 'request_id': 'req-7e3b015a-44ef-4ace-b49b-142df9953ed1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:23.327477 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] The token cache is being invalidate because project 6c4287a54fa04a60beb7055cf5552370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:06:23.327869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:23.328510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e3b015a-44ef-4ace-b49b-142df9953ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:23.330044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1174/2321] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:06:23 2022] DELETE /identity/v3/projects/6c4287a54fa04a60beb7055cf5552370 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:23.334984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82afd00-5dab-47ce-9b2f-3579aa3ef46d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.335294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82afd00-5dab-47ce-9b2f-3579aa3ef46d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.335599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82afd00-5dab-47ce-9b2f-3579aa3ef46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.356417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f82afd00-5dab-47ce-9b2f-3579aa3ef46d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:23.358102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f82afd00-5dab-47ce-9b2f-3579aa3ef46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:23.375739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:23.418184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1148/2322] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:06:23.427326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfccab-ce03-456b-9ac8-f386877ef4ad None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.427660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfccab-ce03-456b-9ac8-f386877ef4ad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.428194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfccab-ce03-456b-9ac8-f386877ef4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.448295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e2dfccab-ce03-456b-9ac8-f386877ef4ad None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:23.449599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2dfccab-ce03-456b-9ac8-f386877ef4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:23.457504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1175/2323] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:06:23.461703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82afd00-5dab-47ce-9b2f-3579aa3ef46d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.471628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82afd00-5dab-47ce-9b2f-3579aa3ef46d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.476108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.476694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.476970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.477237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.477864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:23.478161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.479758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.491620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1149/2324] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:23.495781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfccab-ce03-456b-9ac8-f386877ef4ad None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.504883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfccab-ce03-456b-9ac8-f386877ef4ad None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.509502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.510095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.510373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.510648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.511272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.513229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.543647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1176/2325] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:06:23 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:06:23.549111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.562391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc0d78-daae-447a-841f-683f8cb6eb25 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.624163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.624778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.625056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.625327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.625949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.627885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.648773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1150/2326] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:06:23 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:06:23.653102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.663690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044e3ed-fca3-4aaa-9880-8d689e6571c4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.668397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.669072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.669434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.669775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.670510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.672713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.682601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1177/2327] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:06:23.687109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.697162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f28b-a042-4bb7-9c17-ec74c3b2f1cd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.702253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2b2fc-40d1-4196-9107-1635d0df3fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.702845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2b2fc-40d1-4196-9107-1635d0df3fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.703121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2b2fc-40d1-4196-9107-1635d0df3fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.703447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2b2fc-40d1-4196-9107-1635d0df3fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.704109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2b2fc-40d1-4196-9107-1635d0df3fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.705713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2b2fc-40d1-4196-9107-1635d0df3fbe admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.717525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1151/2328] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:06:23.722405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.733164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b51ff-c4e9-4aee-bdb4-8c8468cd2ab8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.738550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.739210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.739510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.739788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] PATH_INFO: `/v3/projects/1ad9c364fe59404e940c5702c0aecd1d/users/1a25d4383d6041ca970fa8286e98f095/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.754637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad9c364fe59404e940c5702c0aecd1d, user_id=1a25d4383d6041ca970fa8286e98f095, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.757312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.771332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1178/2329] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:06:23 2022] PUT /identity/v3/projects/1ad9c364fe59404e940c5702c0aecd1d/users/1a25d4383d6041ca970fa8286e98f095/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:23.782333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d9eb-76e1-4f8f-a51b-47f2b59ff6db None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.782696 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d9eb-76e1-4f8f-a51b-47f2b59ff6db None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.783045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d9eb-76e1-4f8f-a51b-47f2b59ff6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.812652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-33e5d9eb-76e1-4f8f-a51b-47f2b59ff6db None None] MFA Rules not processed for user `1a25d4383d6041ca970fa8286e98f095`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:23.813885 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-33e5d9eb-76e1-4f8f-a51b-47f2b59ff6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:23.886411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1152/2330] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:06:23.897501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.899285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:23.921609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:23.959833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c4078-c48c-41cf-8f9c-c5a33620bb65 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.963782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075363cb-eb0b-4863-b1fd-3da302db9391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.964274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075363cb-eb0b-4863-b1fd-3da302db9391 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.964536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075363cb-eb0b-4863-b1fd-3da302db9391 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.964793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075363cb-eb0b-4863-b1fd-3da302db9391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.973407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075363cb-eb0b-4863-b1fd-3da302db9391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.974910 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075363cb-eb0b-4863-b1fd-3da302db9391 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.984541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1179/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:06:33.882644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:33.980162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=17 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:36.251205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5d9eb-76e1-4f8f-a51b-47f2b59ff6db None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.315038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5d9eb-76e1-4f8f-a51b-47f2b59ff6db None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.319620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.320252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.320538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.320815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] PATH_INFO: `/v3/users/1a25d4383d6041ca970fa8286e98f095` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.321973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a25d4383d6041ca970fa8286e98f095)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.323923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.370810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a25d4383d6041ca970fa8286e98f095'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.371569 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a25d4383d6041ca970fa8286e98f095'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.379712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a25d4383d6041ca970fa8286e98f095'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.391625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a25d4383d6041ca970fa8286e98f095'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.392269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a25d4383d6041ca970fa8286e98f095'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.405303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1153/2332] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:06:36 2022] DELETE /identity/v3/users/1a25d4383d6041ca970fa8286e98f095 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:36.409865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.410365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.410484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.429818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:36.431005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:36.488883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1180/2333] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:06:36.498816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.500244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:36.555724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b302d-906d-45ac-ab4b-64aeadb3904d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.559262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423f054-2399-4f19-99e0-2cfd99ea2797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.559897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423f054-2399-4f19-99e0-2cfd99ea2797 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.560682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423f054-2399-4f19-99e0-2cfd99ea2797 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.560952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423f054-2399-4f19-99e0-2cfd99ea2797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.569641 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423f054-2399-4f19-99e0-2cfd99ea2797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.571049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423f054-2399-4f19-99e0-2cfd99ea2797 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.580805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1154/2334] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:06:36.844730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.847362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:36.868350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc11656-46bc-440c-9287-f32daa22ef66 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.882438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.883094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.883379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.883700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] PATH_INFO: `/v3/projects/1ad9c364fe59404e940c5702c0aecd1d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.884846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad9c364fe59404e940c5702c0aecd1d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.889047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.941155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad9c364fe59404e940c5702c0aecd1d', 'request_id': 'req-04c6089e-bf7d-4a95-8d55-eb4c5128502a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.947509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad9c364fe59404e940c5702c0aecd1d', 'request_id': 'req-04c6089e-bf7d-4a95-8d55-eb4c5128502a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.953155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad9c364fe59404e940c5702c0aecd1d', 'request_id': 'req-04c6089e-bf7d-4a95-8d55-eb4c5128502a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.964743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] The token cache is being invalidate because project 1ad9c364fe59404e940c5702c0aecd1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:06:36.965067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.965730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-04c6089e-bf7d-4a95-8d55-eb4c5128502a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:36.967760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1181/2335] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:06:36 2022] DELETE /identity/v3/projects/1ad9c364fe59404e940c5702c0aecd1d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:36.972799 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e324d-5ec3-4bc5-ad88-c7dc169e2f8b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.973129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e324d-5ec3-4bc5-ad88-c7dc169e2f8b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.973473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e324d-5ec3-4bc5-ad88-c7dc169e2f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.994258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-167e324d-5ec3-4bc5-ad88-c7dc169e2f8b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:36.995427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-167e324d-5ec3-4bc5-ad88-c7dc169e2f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:37.012281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:37.055778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1155/2336] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:06:37.064703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f5ef-386f-4ee8-a432-464478550ce3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.064989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f5ef-386f-4ee8-a432-464478550ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.065263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f5ef-386f-4ee8-a432-464478550ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.084808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9e29f5ef-386f-4ee8-a432-464478550ce3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:37.086023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9e29f5ef-386f-4ee8-a432-464478550ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:37.094550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1182/2337] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:06:37.098601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e324d-5ec3-4bc5-ad88-c7dc169e2f8b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.108182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e324d-5ec3-4bc5-ad88-c7dc169e2f8b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.117090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.117689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.117968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.118240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.118997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:37.119304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.120999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.133259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1156/2338] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:06:37.137731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f5ef-386f-4ee8-a432-464478550ce3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.148845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f5ef-386f-4ee8-a432-464478550ce3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.153593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.154247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.154545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.154817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.155445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.157371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.184472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1183/2339] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:06:37 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:06:37.189059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.199340 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d3744-bc93-49e1-a21a-6331e2d9cee4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.256157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.256777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.257045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.257312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.257942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.259874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.281822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1157/2340] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:06:37 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:06:37.286061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.296732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e6bd8-0daf-4c2e-813d-bc28c7fd7015 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.301476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.302073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.302352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.302624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.303544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.305143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.315543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1184/2341] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:06:37.319372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.329622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face7fb-b738-4cfc-bc06-e99aab580cc1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.334343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a593f-2815-4214-9d34-f299b34daf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.334939 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a593f-2815-4214-9d34-f299b34daf4a admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.335215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a593f-2815-4214-9d34-f299b34daf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.335482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a593f-2815-4214-9d34-f299b34daf4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.336182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a593f-2815-4214-9d34-f299b34daf4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.337758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a593f-2815-4214-9d34-f299b34daf4a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.348784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1158/2342] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:06:37.353499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.364729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974adbb-c15f-44c4-ac51-780bfdcd17bd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.369612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.370250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.370526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.370794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] PATH_INFO: `/v3/projects/56f99bdf91e544ee83f1316d0bd6fc9e/users/e7e32d229345414e8835a407b82f57e2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.385489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f99bdf91e544ee83f1316d0bd6fc9e, user_id=e7e32d229345414e8835a407b82f57e2, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.388145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.400331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1185/2343] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:06:37 2022] PUT /identity/v3/projects/56f99bdf91e544ee83f1316d0bd6fc9e/users/e7e32d229345414e8835a407b82f57e2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:37.410728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b95b0c-fd1d-43b9-8a5d-cc0a74ad7b75 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.411011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b95b0c-fd1d-43b9-8a5d-cc0a74ad7b75 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.411281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b95b0c-fd1d-43b9-8a5d-cc0a74ad7b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.439966 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f2b95b0c-fd1d-43b9-8a5d-cc0a74ad7b75 None None] MFA Rules not processed for user `e7e32d229345414e8835a407b82f57e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:37.441168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f2b95b0c-fd1d-43b9-8a5d-cc0a74ad7b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:37.503481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1159/2344] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:06:37.514391 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.516093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:37.537139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:37.573356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfbab7-6e52-4ccb-b86e-24d245d4085b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.577019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceee10-c460-4a2f-a00e-1294f4c87c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.577619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceee10-c460-4a2f-a00e-1294f4c87c4d service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.577898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceee10-c460-4a2f-a00e-1294f4c87c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.578167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceee10-c460-4a2f-a00e-1294f4c87c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.586309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceee10-c460-4a2f-a00e-1294f4c87c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.587763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceee10-c460-4a2f-a00e-1294f4c87c4d service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.596372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1186/2345] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:06:47.500608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:47.592456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=17 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:59.156832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b95b0c-fd1d-43b9-8a5d-cc0a74ad7b75 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:59.227451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b95b0c-fd1d-43b9-8a5d-cc0a74ad7b75 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:59.231905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:59.232523 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:59.232800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:59.233070 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] PATH_INFO: `/v3/users/e7e32d229345414e8835a407b82f57e2` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:59.234303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e32d229345414e8835a407b82f57e2)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:59.236310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:59.283874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e32d229345414e8835a407b82f57e2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.284784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e32d229345414e8835a407b82f57e2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.291979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e32d229345414e8835a407b82f57e2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.303921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e32d229345414e8835a407b82f57e2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.304563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e32d229345414e8835a407b82f57e2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.316862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1160/2346] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:06:59 2022] DELETE /identity/v3/users/e7e32d229345414e8835a407b82f57e2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:59.321470 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:59.321946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:59.322057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:59.341866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:59.343064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:59.404168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1187/2347] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:06:59.414448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:59.415878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:59.476416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da16081-1925-4cfa-b8d4-4ef56b930154 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:59.480199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:59.480810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:59.481100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:59.481373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:59.491360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:59.493017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:59.503449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1161/2348] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:06:59.784954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:59.786819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:59.802661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd6966-9239-47ca-aeb1-8b6c78e0dd74 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:59.807853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:59.808498 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:59.809104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:59.809790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] PATH_INFO: `/v3/projects/56f99bdf91e544ee83f1316d0bd6fc9e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:59.811136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f99bdf91e544ee83f1316d0bd6fc9e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:59.813724 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:59.866810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f99bdf91e544ee83f1316d0bd6fc9e', 'request_id': 'req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.873767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f99bdf91e544ee83f1316d0bd6fc9e', 'request_id': 'req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.881507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f99bdf91e544ee83f1316d0bd6fc9e', 'request_id': 'req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.891121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:59.892812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:59.899893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] The token cache is being invalidate because project 56f99bdf91e544ee83f1316d0bd6fc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:06:59.900235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.900918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0e9389ce-21a6-4ba5-95b2-0e830bb359a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:06:59.902648 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1188/2349] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:06:59 2022] DELETE /identity/v3/projects/56f99bdf91e544ee83f1316d0bd6fc9e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:59.907770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632009-8340-4445-9cac-c5e2e24b3f1d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:59.908047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632009-8340-4445-9cac-c5e2e24b3f1d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:59.908312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632009-8340-4445-9cac-c5e2e24b3f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:59.912066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f70873-e360-41e6-b9a7-3344a5147e89 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:59.927995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b8632009-8340-4445-9cac-c5e2e24b3f1d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:59.929454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b8632009-8340-4445-9cac-c5e2e24b3f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:59.931561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:59.982505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:59.983146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:59.983431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:59.983742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] PATH_INFO: `/v3/users/cf0dc39479d1435f902e10cb6827f5e4` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:59.985410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0dc39479d1435f902e10cb6827f5e4)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:59.987383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:59.995892 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1189/2350] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:00.004908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23573b75-158c-43d4-8431-702c09bfd0ad None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.005193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23573b75-158c-43d4-8431-702c09bfd0ad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.005460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23573b75-158c-43d4-8431-702c09bfd0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.025361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-23573b75-158c-43d4-8431-702c09bfd0ad None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:00.026740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23573b75-158c-43d4-8431-702c09bfd0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:00.038783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1190/2351] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:07:00.039412 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0dc39479d1435f902e10cb6827f5e4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.039743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23573b75-158c-43d4-8431-702c09bfd0ad None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.040334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0dc39479d1435f902e10cb6827f5e4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.048594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0dc39479d1435f902e10cb6827f5e4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.049207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23573b75-158c-43d4-8431-702c09bfd0ad None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.054273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.054803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.055080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.055355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.056029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:07:00.056330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.057916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.065001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0dc39479d1435f902e10cb6827f5e4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.066793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0dc39479d1435f902e10cb6827f5e4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.071030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1191/2352] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:07:00.075931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.077866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:00.087196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1162/2353] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:06:59 2022] DELETE /identity/v3/users/cf0dc39479d1435f902e10cb6827f5e4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:07:00.088100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:00.093077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d26f54-d449-4278-9408-b38553f6e6e7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.162301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.162913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.163193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.163464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.164146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.166042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.197286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1192/2354] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:07:00 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:07:00.201425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.211931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401fbda-d861-4ead-9c22-484ff059f72c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.304142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.305240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.305240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.305427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.306112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.308385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.335996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1163/2355] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:07:00 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:07:00.337856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.350476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f0bf1-c630-4cbb-ae7a-2c3f2c2495ee admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.356363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.357099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.357483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.357943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.358634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.361153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.380559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1193/2356] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:07:00.380949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.391598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636adf57-ee33-40bb-b8f9-0cd8a67f94ef admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.394641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.400589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:00.403901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.404616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.405065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.405338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.406665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.408955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.423687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1164/2357] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:07:00.428508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.429834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653137b-cded-4821-94c6-e063c2551c8a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.436706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.437649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.437931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.438198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] PATH_INFO: `/v3/projects/70d39d21d2844c3599629db4abdc1600` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.439900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70d39d21d2844c3599629db4abdc1600)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.442120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.442639 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fc96f-f79d-40e5-8844-86b8c005de45 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.447733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53509227-1fb8-4dbe-8ae8-1ce351b175e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.448393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53509227-1fb8-4dbe-8ae8-1ce351b175e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.448677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53509227-1fb8-4dbe-8ae8-1ce351b175e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.448952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53509227-1fb8-4dbe-8ae8-1ce351b175e1 admin admin] PATH_INFO: `/v3/projects/52565b3ddbe9435e8112f134f6c5d496/users/1bdf9e74be004e8ab24b9a15bb8904d2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.465508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53509227-1fb8-4dbe-8ae8-1ce351b175e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52565b3ddbe9435e8112f134f6c5d496, user_id=1bdf9e74be004e8ab24b9a15bb8904d2, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.468788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53509227-1fb8-4dbe-8ae8-1ce351b175e1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.472817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:00.486474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1165/2358] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:07:00 2022] PUT /identity/v3/projects/52565b3ddbe9435e8112f134f6c5d496/users/1bdf9e74be004e8ab24b9a15bb8904d2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:07:00.491741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc557d-86b5-49b4-8b52-34190ba72e2a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.492162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc557d-86b5-49b4-8b52-34190ba72e2a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.492787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc557d-86b5-49b4-8b52-34190ba72e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.493559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70d39d21d2844c3599629db4abdc1600', 'request_id': 'req-b15f8312-d895-434f-8203-35469daf5429'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.500370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70d39d21d2844c3599629db4abdc1600', 'request_id': 'req-b15f8312-d895-434f-8203-35469daf5429'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.506481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70d39d21d2844c3599629db4abdc1600', 'request_id': 'req-b15f8312-d895-434f-8203-35469daf5429'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.511864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8bcc557d-86b5-49b4-8b52-34190ba72e2a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:00.513438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8bcc557d-86b5-49b4-8b52-34190ba72e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:00.520435 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] The token cache is being invalidate because project 70d39d21d2844c3599629db4abdc1600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:07:00.520727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.521350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:07:00.522844 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1194/2359] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:07:00 2022] DELETE /identity/v3/projects/70d39d21d2844c3599629db4abdc1600 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:07:00.580082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1166/2360] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:07:00.590414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:00.591970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:00.655536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f8312-d895-434f-8203-35469daf5429 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:00.659946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:00.660583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:00.660864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:00.661139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:00.671247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:00.672802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:00.683678 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1195/2361] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:07:05.841443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa709b1b-3e8b-4208-811f-e348a0834314 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:05.842001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa709b1b-3e8b-4208-811f-e348a0834314 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:05.842001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa709b1b-3e8b-4208-811f-e348a0834314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:05.916877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-fa709b1b-3e8b-4208-811f-e348a0834314 None None] MFA Rules not processed for user `1bdf9e74be004e8ab24b9a15bb8904d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:05.918656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-fa709b1b-3e8b-4208-811f-e348a0834314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:06.105274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1167/2362] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:06.111919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:06.113376 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:06.133706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:06.144064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cda447-b9ec-414b-b88a-b00ccbf68298 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:06.157211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:06.157838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:06.158114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:06.158387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:06.159030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:06.160994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:06.214862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1196/2363] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:07:06 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:07:06.219439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa709b1b-3e8b-4208-811f-e348a0834314 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:06.235969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa709b1b-3e8b-4208-811f-e348a0834314 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:06.320749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:06.321507 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:06.321935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:06.322218 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:06.322945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:06.324967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:06.358689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1168/2364] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:07:06 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:07:06.359367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:06.371454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf009f33-9ce4-4909-a8a3-5ae20b45bdd7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:06.377515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:06.378476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:06.378476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:06.378815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:06.379603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:06.381176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:06.400736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:06.409366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1197/2365] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:07:06.413109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce9d2-6f2c-4b3a-a742-39fdbdbc9d5b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:06.419858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:06.420710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:06.421712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:06.421991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] PATH_INFO: `/v3/projects/16bf50d1c8264278bbe680d5096d3ce0/users/11a237c3d68c4634bfc8f56b01361582/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:06.435777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:06.439958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] RBAC: Authorizing `identity:create_grant(project_id=16bf50d1c8264278bbe680d5096d3ce0, user_id=11a237c3d68c4634bfc8f56b01361582, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:06.443477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:06.466293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1169/2366] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:07:06 2022] PUT /identity/v3/projects/16bf50d1c8264278bbe680d5096d3ce0/users/11a237c3d68c4634bfc8f56b01361582/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:07:06.467570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:06.479006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbcde6-c79a-43fb-a787-9a1710494389 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:06.591268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223cd34-8ccb-4e80-a186-a4ed9e2e7676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:06.592027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cd34-8ccb-4e80-a186-a4ed9e2e7676 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:06.592321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cd34-8ccb-4e80-a186-a4ed9e2e7676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:06.592600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cd34-8ccb-4e80-a186-a4ed9e2e7676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:06.594503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223cd34-8ccb-4e80-a186-a4ed9e2e7676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:06.596220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223cd34-8ccb-4e80-a186-a4ed9e2e7676 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:06.608979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1198/2367] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:07:06.615142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:06.645094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da41a63-c466-4264-ae8a-f22900b7739c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:06.665361 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d5595-3a4b-45fc-aa8b-68ae97898f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:06.666052 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d5595-3a4b-45fc-aa8b-68ae97898f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:06.666337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d5595-3a4b-45fc-aa8b-68ae97898f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:06.666612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d5595-3a4b-45fc-aa8b-68ae97898f07 admin admin] PATH_INFO: `/v3/projects/16bf50d1c8264278bbe680d5096d3ce0/users/11a237c3d68c4634bfc8f56b01361582/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:06.673075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d5595-3a4b-45fc-aa8b-68ae97898f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16bf50d1c8264278bbe680d5096d3ce0, user_id=11a237c3d68c4634bfc8f56b01361582, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:06.676054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d5595-3a4b-45fc-aa8b-68ae97898f07 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:06.705355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1170/2368] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:07:06 2022] PUT /identity/v3/projects/16bf50d1c8264278bbe680d5096d3ce0/users/11a237c3d68c4634bfc8f56b01361582/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:07:08.385135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d899e3-6366-4020-b098-d42dd34fac82 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:08.386107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d899e3-6366-4020-b098-d42dd34fac82 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:08.386549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d899e3-6366-4020-b098-d42dd34fac82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:08.450583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-52d899e3-6366-4020-b098-d42dd34fac82 None None] MFA Rules not processed for user `e24e6fb4c69d48f7b2972b53fb8ea4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:08.453421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-52d899e3-6366-4020-b098-d42dd34fac82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:08.585883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1199/2369] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:08.889112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511257-b6bd-403f-8607-701ce8e0de15 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:08.889951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511257-b6bd-403f-8607-701ce8e0de15 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:08.890397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511257-b6bd-403f-8607-701ce8e0de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:08.917514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-97511257-b6bd-403f-8607-701ce8e0de15 None None] MFA Rules not processed for user `85cd8a90230b4c69886534be67acd943`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:08.919119 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-97511257-b6bd-403f-8607-701ce8e0de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:09.039551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1171/2370] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:13.610165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:13.610684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:13.610797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:13.675190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] MFA Rules not processed for user `11a237c3d68c4634bfc8f56b01361582`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:13.676536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:13.766661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd89d0e-a985-405a-b5bf-23f7095c9262 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:13.767613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd89d0e-a985-405a-b5bf-23f7095c9262 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:13.768061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd89d0e-a985-405a-b5bf-23f7095c9262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:13.784064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:13.803667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1200/2371] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:13.808497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6bd89d0e-a985-405a-b5bf-23f7095c9262 None None] MFA Rules not processed for user `eaef8f72768e4d1daf21257fdaf580d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:13.809750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6bd89d0e-a985-405a-b5bf-23f7095c9262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:13.811099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:13.812704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:13.896003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1172/2372] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:13.901244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca734b-7718-42e1-ad09-8fd91efc748a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:13.905793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:13.906446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:13.906697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:13.906970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:14.028807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:14.031065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:14.048505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1201/2373] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:07:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:07:14.305580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:14.305870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:14.306152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:14.332547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:14.334056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:14.418747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1173/2374] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:07:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:14.489054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:14.490573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:14.576398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6f7af-b061-4ee0-9826-3a4e49c18c86 service glance] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:14.586460 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:14.587078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:14.587368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:14.588595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:14.598565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:14.600382 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:14.614999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1202/2375] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:07:22.144132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:22.145556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:22.190639 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:22.215764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966512b-925a-4204-bbeb-e7a7c3c5bef3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:22.220094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be63256-65c0-4c6f-a66a-9e0bff22b7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:22.220714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63256-65c0-4c6f-a66a-9e0bff22b7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:22.221045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63256-65c0-4c6f-a66a-9e0bff22b7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:22.221328 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63256-65c0-4c6f-a66a-9e0bff22b7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:22.223058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7be63256-65c0-4c6f-a66a-9e0bff22b7b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:22.311270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be63256-65c0-4c6f-a66a-9e0bff22b7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:22.312780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be63256-65c0-4c6f-a66a-9e0bff22b7b6 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:22.325775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1174/2376] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:07:24.607798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:24.925605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:24.938400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47644dbd-28c2-44e2-8cc1-e3d13789a235 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:24.942430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a9226-dc01-457c-94d4-c9cea073ad86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:24.943044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a9226-dc01-457c-94d4-c9cea073ad86 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:24.943326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a9226-dc01-457c-94d4-c9cea073ad86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:24.943764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a9226-dc01-457c-94d4-c9cea073ad86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:24.945522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-741a9226-dc01-457c-94d4-c9cea073ad86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:24.963047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a9226-dc01-457c-94d4-c9cea073ad86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:24.964705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a9226-dc01-457c-94d4-c9cea073ad86 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:25.016792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1203/2377] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:07:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:07:26.533083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a4be7-5680-4055-8296-564657101454 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:26.533691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a4be7-5680-4055-8296-564657101454 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:26.533691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a4be7-5680-4055-8296-564657101454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:26.552687 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-233a4be7-5680-4055-8296-564657101454 None None] MFA Rules not processed for user `1bdf9e74be004e8ab24b9a15bb8904d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:26.554125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-233a4be7-5680-4055-8296-564657101454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:26.560843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1175/2378] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:07:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:07:30.062277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7269047-4dee-4996-8346-93f565486b8d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:30.062777 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7269047-4dee-4996-8346-93f565486b8d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:30.062965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7269047-4dee-4996-8346-93f565486b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:30.087750 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e7269047-4dee-4996-8346-93f565486b8d None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:07:30.089231 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e7269047-4dee-4996-8346-93f565486b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:30.098181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1204/2379] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:07:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:07:30.113109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a4be7-5680-4055-8296-564657101454 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:30.125323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a4be7-5680-4055-8296-564657101454 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:30.130381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:30.131004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:30.131263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:30.131573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:30.143552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:30.145679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:30.155789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1176/2380] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:07:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:07:40.081835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=9 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:40.151341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=48 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:47.811050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7269047-4dee-4996-8346-93f565486b8d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:07:47.812494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e7269047-4dee-4996-8346-93f565486b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:47.866789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7269047-4dee-4996-8346-93f565486b8d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:07:47.870348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:07:47.870946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:07:47.871224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:07:47.871505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:07:47.873206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:07:47.883693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:07:47.885108 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:07:47.894338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1205/2381] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:07:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:07:57.890645 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=26 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:06.432708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:06.434332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:06.454484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28219001-b5c7-4b15-b624-3b5a7eedb07e service swift] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:06.466656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:06.469008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:06.469864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:06.470499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:06.472632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:06.486568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:06.488538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:06.500433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:06.507692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1177/2382] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:06.519097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3362054-3fb3-4307-a891-cac5994dfc9c service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:06.524864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:06.526100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:06.526743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:06.527240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:06.541962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:06.543446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:06.555095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1206/2383] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:08:08.614572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:08.616024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:08.630584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3eda1e-4f66-4e50-84de-23d5c765020b service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:08.637336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9aed7-11ef-44d6-aa88-ec8f7aa9cbc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:08.638059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9aed7-11ef-44d6-aa88-ec8f7aa9cbc2 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:08.638405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9aed7-11ef-44d6-aa88-ec8f7aa9cbc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:08.638782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9aed7-11ef-44d6-aa88-ec8f7aa9cbc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:08.641473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e9f9aed7-11ef-44d6-aa88-ec8f7aa9cbc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:08.654023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9aed7-11ef-44d6-aa88-ec8f7aa9cbc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:08.655984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9aed7-11ef-44d6-aa88-ec8f7aa9cbc2 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:08.667743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1178/2384] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:10.318114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:10.334069 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce03b-3986-46fe-8467-709744945ddc service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:10.345941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:10.347194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:10.347602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:10.348055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:10.351729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:10.365628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:10.367705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:10.379045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1207/2385] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:11.461878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6b0ae-86fc-4e45-aec9-db5dfbbfbdfb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:11.462449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6b0ae-86fc-4e45-aec9-db5dfbbfbdfb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:11.462449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6b0ae-86fc-4e45-aec9-db5dfbbfbdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:11.480897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-dab6b0ae-86fc-4e45-aec9-db5dfbbfbdfb None None] MFA Rules not processed for user `3cbefab6dddb45c381158126d203f66b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:11.482109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-dab6b0ae-86fc-4e45-aec9-db5dfbbfbdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:11.540294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1179/2386] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:11.876083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:11.887262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38923-02c7-4e09-b335-d80651cea5f9 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:11.891895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:11.893145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:11.893619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:11.894054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:11.896422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:11.907158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:11.908732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:11.930501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1208/2387] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:12.385255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6b0ae-86fc-4e45-aec9-db5dfbbfbdfb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:12.399895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6b0ae-86fc-4e45-aec9-db5dfbbfbdfb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:12.411309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:12.412364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:12.413499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:12.413985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:12.417254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:12.433100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:12.435670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:12.453105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1180/2388] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:12.454956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:12.456482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:12.551168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0881cc-a201-4194-a29b-4b78c514e67a service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:12.556707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:12.557388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:12.557665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:12.557934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:12.559682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:12.571024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:12.572584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:12.583467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1209/2389] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:13.888522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:13.902377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a1983-36b3-49a0-ae31-bf298bf88b31 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:13.908557 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:13.909171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:13.909454 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:13.909731 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:13.911934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:13.997346 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:13.999380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:14.009500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1181/2390] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:15.682094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:15.696758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f728b6-9c86-4186-9195-5e4b770bc9ca service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:15.734520 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:15.736245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:15.737013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:15.737454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:15.744649 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:15.768312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:15.770027 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:15.788356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1210/2391] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:17.739238 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:17.740782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:17.753032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad498f6d-d83f-41a9-9f32-ed5a33868071 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:17.757668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:17.758292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:17.758599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:17.758870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] PATH_INFO: `/v3/users/e24e6fb4c69d48f7b2972b53fb8ea4a1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:17.760073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24e6fb4c69d48f7b2972b53fb8ea4a1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:17.761978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:17.811944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24e6fb4c69d48f7b2972b53fb8ea4a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:17.812522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24e6fb4c69d48f7b2972b53fb8ea4a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:17.817740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:17.819738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24e6fb4c69d48f7b2972b53fb8ea4a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:17.835764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24e6fb4c69d48f7b2972b53fb8ea4a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:17.836466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b292203e-e742-4f2f-b9f6-6168e6803ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24e6fb4c69d48f7b2972b53fb8ea4a1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:17.850691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1182/2392] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:17 2022] DELETE /identity/v3/users/e24e6fb4c69d48f7b2972b53fb8ea4a1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:18.029316 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.042491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46603eeb-0982-4ae5-80cd-fcf1d8867f26 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.105959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.106631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.106912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.107188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] PATH_INFO: `/v3/projects/d688ef3e358c4822823707982fe4f654` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.108856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d688ef3e358c4822823707982fe4f654)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.110893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.154035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:18.161595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd688ef3e358c4822823707982fe4f654', 'request_id': 'req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:18.167952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd688ef3e358c4822823707982fe4f654', 'request_id': 'req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:18.173852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd688ef3e358c4822823707982fe4f654', 'request_id': 'req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:18.187700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] The token cache is being invalidate because project d688ef3e358c4822823707982fe4f654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:18.187990 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:18.188667 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3816da7b-63a1-4ddc-ba8f-2bcc0f30191a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:18.190171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1211/2393] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:18 2022] DELETE /identity/v3/projects/d688ef3e358c4822823707982fe4f654 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:18.195084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53783c96-a75a-47e1-8ef5-093a50ba0943 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.195374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53783c96-a75a-47e1-8ef5-093a50ba0943 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.195699 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53783c96-a75a-47e1-8ef5-093a50ba0943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.215984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-53783c96-a75a-47e1-8ef5-093a50ba0943 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:18.217240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-53783c96-a75a-47e1-8ef5-093a50ba0943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:18.276928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1183/2394] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:18.284633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97600c53-2d69-4df2-a53d-e00dcc4e0826 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.284917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97600c53-2d69-4df2-a53d-e00dcc4e0826 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.285185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97600c53-2d69-4df2-a53d-e00dcc4e0826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.304471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-97600c53-2d69-4df2-a53d-e00dcc4e0826 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:18.306016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-97600c53-2d69-4df2-a53d-e00dcc4e0826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:18.314268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1212/2395] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:18.318419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53783c96-a75a-47e1-8ef5-093a50ba0943 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.328387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53783c96-a75a-47e1-8ef5-093a50ba0943 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.332879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.333963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.334239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.334499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.335130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:08:18.335425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.337039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.348531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1184/2396] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:08:18.352581 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97600c53-2d69-4df2-a53d-e00dcc4e0826 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.361716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97600c53-2d69-4df2-a53d-e00dcc4e0826 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.367175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.367795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.368071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.368369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.369002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.370877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.400987 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1213/2397] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:18 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:18.404974 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.415661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b85290-f6b5-4987-8910-756c33fd7918 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.473620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.474234 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.474543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.474789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.475434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.477388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.499861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1185/2398] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:18 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:18.504096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.515550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c5e24-3b64-4fad-a0ce-188e0af7b010 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.520344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.520941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.521217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.521483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.522122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.523728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.535394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1214/2399] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:18.539395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.549998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04fc77-da9f-422f-b7fc-5581521dc52e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.554896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11dc46-06a0-4a35-b549-69ab1c559e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.555481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11dc46-06a0-4a35-b549-69ab1c559e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.555786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11dc46-06a0-4a35-b549-69ab1c559e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.556055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11dc46-06a0-4a35-b549-69ab1c559e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.556681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11dc46-06a0-4a35-b549-69ab1c559e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.558251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11dc46-06a0-4a35-b549-69ab1c559e40 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.569166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1186/2400] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:18.573040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.582834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a8e1-c128-48bc-88db-c86ab5926e18 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.587420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.588086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.588494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.588764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] PATH_INFO: `/v3/projects/15c678864a9f406fb3eafc98831e5ce6/users/384cf0d4f7e34642baa66767bbe8a4a5/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.603126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c678864a9f406fb3eafc98831e5ce6, user_id=384cf0d4f7e34642baa66767bbe8a4a5, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.605818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.618813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1215/2401] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:18 2022] PUT /identity/v3/projects/15c678864a9f406fb3eafc98831e5ce6/users/384cf0d4f7e34642baa66767bbe8a4a5/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:18.630394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d9f7a-99a7-4515-90ab-317dcfb0b446 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.630677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d9f7a-99a7-4515-90ab-317dcfb0b446 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.630946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d9f7a-99a7-4515-90ab-317dcfb0b446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.659650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-611d9f7a-99a7-4515-90ab-317dcfb0b446 None None] MFA Rules not processed for user `384cf0d4f7e34642baa66767bbe8a4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:18.660879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-611d9f7a-99a7-4515-90ab-317dcfb0b446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:18.687514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:18.726707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1187/2402] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:18 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:18.731170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.794370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219fd0f-621f-4fc3-9c98-f1cc75f13051 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.799563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.800162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.800443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.800713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.801351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.803240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.830204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1216/2403] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:18 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:18.834138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d9f7a-99a7-4515-90ab-317dcfb0b446 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.844755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d9f7a-99a7-4515-90ab-317dcfb0b446 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.900022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.900628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.900908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.901182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.901807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.903734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.924463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1188/2404] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:18 2022] POST /identity/v3/users => generated 308 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:18.928400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.938637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f33b2-c804-439f-9f06-964f573e88c0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.943161 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.943798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.944075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.944345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.944974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.946835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:18.957043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1217/2405] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:18.960903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:18.971251 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376f600-e248-4328-a976-a57ebde74455 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:18.975980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:18.976618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:18.976899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:18.977174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] PATH_INFO: `/v3/projects/ccf1b8a4a64e456b9c122b41547c1fe0/users/114ec605b66f41aba21b2dd1335fe50f/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:18.993234 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf1b8a4a64e456b9c122b41547c1fe0, user_id=114ec605b66f41aba21b2dd1335fe50f, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:18.996016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:19.009237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1189/2406] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:18 2022] PUT /identity/v3/projects/ccf1b8a4a64e456b9c122b41547c1fe0/users/114ec605b66f41aba21b2dd1335fe50f/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:19.013226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:19.022980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d9720-ddf7-4fda-9cdb-975fa4649a83 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:19.056771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:19.080664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0191ef-273a-44a8-8a6c-e346a7afb463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:19.081202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0191ef-273a-44a8-8a6c-e346a7afb463 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:19.081484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0191ef-273a-44a8-8a6c-e346a7afb463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:19.081752 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0191ef-273a-44a8-8a6c-e346a7afb463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:19.082383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0191ef-273a-44a8-8a6c-e346a7afb463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:19.084025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0191ef-273a-44a8-8a6c-e346a7afb463 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:19.093963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1218/2407] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:19.098596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:19.109480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92413a-d797-4ac5-bae4-11ffab2cead3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:19.114147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:19.114790 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:19.115074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:19.115350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] PATH_INFO: `/v3/projects/ccf1b8a4a64e456b9c122b41547c1fe0/users/114ec605b66f41aba21b2dd1335fe50f/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:19.117268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf1b8a4a64e456b9c122b41547c1fe0, user_id=114ec605b66f41aba21b2dd1335fe50f, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:19.119935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:19.129852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1190/2408] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:19 2022] PUT /identity/v3/projects/ccf1b8a4a64e456b9c122b41547c1fe0/users/114ec605b66f41aba21b2dd1335fe50f/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:19.139920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4a5d1-0252-4502-9769-ec4146444aa1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:19.140213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4a5d1-0252-4502-9769-ec4146444aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:19.140490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4a5d1-0252-4502-9769-ec4146444aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:19.167729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3cf4a5d1-0252-4502-9769-ec4146444aa1 None None] MFA Rules not processed for user `114ec605b66f41aba21b2dd1335fe50f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:19.168938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3cf4a5d1-0252-4502-9769-ec4146444aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:19.240915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1219/2409] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:19 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:19.251768 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:19.253477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:19.327056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79e82e-27c9-4aec-95cc-c6fd23aa97b4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:19.330794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be95ec-109a-4af5-9cec-02d45f6672e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:19.331323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be95ec-109a-4af5-9cec-02d45f6672e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:19.331631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be95ec-109a-4af5-9cec-02d45f6672e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:19.331898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be95ec-109a-4af5-9cec-02d45f6672e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:19.341483 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be95ec-109a-4af5-9cec-02d45f6672e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:19.343603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be95ec-109a-4af5-9cec-02d45f6672e8 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:19.353408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1191/2410] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:19 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:19.507257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4a5d1-0252-4502-9769-ec4146444aa1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:19.573653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4a5d1-0252-4502-9769-ec4146444aa1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:19.578869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:19.579530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:19.579845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:19.580120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] PATH_INFO: `/v3/users/384cf0d4f7e34642baa66767bbe8a4a5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:19.581311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=384cf0d4f7e34642baa66767bbe8a4a5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:19.583194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:19.641204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384cf0d4f7e34642baa66767bbe8a4a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:19.653459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384cf0d4f7e34642baa66767bbe8a4a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:19.661239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:19.674112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384cf0d4f7e34642baa66767bbe8a4a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:19.675347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384cf0d4f7e34642baa66767bbe8a4a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:19.675731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384cf0d4f7e34642baa66767bbe8a4a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:19.690693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1220/2411] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:19 2022] DELETE /identity/v3/users/384cf0d4f7e34642baa66767bbe8a4a5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:19.695115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:19.695406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:19.695726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:19.717364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:19.718586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:19.737075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:19.784691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1192/2412] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:19.794848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:19.796317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:19.856759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4474a-182a-4ab3-9a4b-94c71690cab0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:19.860687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:19.861292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:19.861570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:19.861842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:19.872182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:19.874129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:19.883463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1221/2413] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:20.165516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:20.167104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:20.182656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736687a2-92ff-4c04-bd97-58e66535b34f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:20.189080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:20.189669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:20.189942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:20.190338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] PATH_INFO: `/v3/projects/15c678864a9f406fb3eafc98831e5ce6` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:20.191467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c678864a9f406fb3eafc98831e5ce6)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:20.193448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:20.243905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c678864a9f406fb3eafc98831e5ce6', 'request_id': 'req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.250396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c678864a9f406fb3eafc98831e5ce6', 'request_id': 'req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.257895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c678864a9f406fb3eafc98831e5ce6', 'request_id': 'req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.268720 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] The token cache is being invalidate because project 15c678864a9f406fb3eafc98831e5ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:20.269016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.269680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.271185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1193/2414] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:20 2022] DELETE /identity/v3/projects/15c678864a9f406fb3eafc98831e5ce6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:20.275204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:20.288928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513c733-a2ef-4a8c-8444-0875e36adba7 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:20.346185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:20.346723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:20.347004 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:20.347281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] PATH_INFO: `/v3/users/114ec605b66f41aba21b2dd1335fe50f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:20.348441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] RBAC: Authorizing `identity:delete_user(user_id=114ec605b66f41aba21b2dd1335fe50f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:20.350347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:20.396204 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114ec605b66f41aba21b2dd1335fe50f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.403052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114ec605b66f41aba21b2dd1335fe50f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.414407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114ec605b66f41aba21b2dd1335fe50f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.415059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114ec605b66f41aba21b2dd1335fe50f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.415679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-52dd0845-eb94-4930-a4e9-d78085d5416a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114ec605b66f41aba21b2dd1335fe50f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.424497 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:20.429302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1222/2415] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:20 2022] DELETE /identity/v3/users/114ec605b66f41aba21b2dd1335fe50f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:20.712713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:20.714145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:20.728163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e9c19-fcc9-4ea7-8faf-a95423ca97d8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:20.790061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:20.790708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:20.790985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:20.791256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] PATH_INFO: `/v3/projects/ccf1b8a4a64e456b9c122b41547c1fe0` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:20.792409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf1b8a4a64e456b9c122b41547c1fe0)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:20.794314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:20.847105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:20.848602 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf1b8a4a64e456b9c122b41547c1fe0', 'request_id': 'req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.858189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf1b8a4a64e456b9c122b41547c1fe0', 'request_id': 'req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.869344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf1b8a4a64e456b9c122b41547c1fe0', 'request_id': 'req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.885020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] The token cache is being invalidate because project ccf1b8a4a64e456b9c122b41547c1fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:20.886118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.886118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-b17c9358-1a85-4bb2-ac9b-a3c60f044acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:20.891239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1194/2416] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:20 2022] DELETE /identity/v3/projects/ccf1b8a4a64e456b9c122b41547c1fe0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:20.892973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817ba03-6a15-4f57-9871-4b63a3f149f4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:20.893275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817ba03-6a15-4f57-9871-4b63a3f149f4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:20.893676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817ba03-6a15-4f57-9871-4b63a3f149f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:20.932212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f817ba03-6a15-4f57-9871-4b63a3f149f4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:20.933446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f817ba03-6a15-4f57-9871-4b63a3f149f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:20.992601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1223/2417] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:21.001077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a76332-b5bf-4b04-9f78-571d803832cd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.001367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a76332-b5bf-4b04-9f78-571d803832cd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.001646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a76332-b5bf-4b04-9f78-571d803832cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.021197 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b4a76332-b5bf-4b04-9f78-571d803832cd None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:21.022396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b4a76332-b5bf-4b04-9f78-571d803832cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:21.030229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1195/2418] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:21.034250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817ba03-6a15-4f57-9871-4b63a3f149f4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:21.043915 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817ba03-6a15-4f57-9871-4b63a3f149f4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:21.049150 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:21.049736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.050013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.050281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.050965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:08:21.051275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:21.052961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:21.065179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1224/2419] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:08:21.069233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a76332-b5bf-4b04-9f78-571d803832cd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:21.078216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a76332-b5bf-4b04-9f78-571d803832cd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:21.082802 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:21.083386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.083712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.084001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.084632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:21.086509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:21.113550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1196/2420] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:21 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:21.117841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:21.128425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0484-1447-4e76-a545-052d38ff5d3d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:21.183758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:21.184360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.184644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.184922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.185558 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:21.187472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:21.209864 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1225/2421] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:21 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:21.213989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:21.224535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79a1c8-f537-4589-8f06-355eac4db634 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:21.229160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:21.229752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.230029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.230296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.230959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:21.233421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:21.243453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1197/2422] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:21.247786 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:21.257853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a17d33-5c3d-4f77-a940-ca3ec9bbd86d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:21.262448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef021a-8e6d-41e3-a32f-24a0c432c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:21.263044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef021a-8e6d-41e3-a32f-24a0c432c755 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.263319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef021a-8e6d-41e3-a32f-24a0c432c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.263635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef021a-8e6d-41e3-a32f-24a0c432c755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.264296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef021a-8e6d-41e3-a32f-24a0c432c755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:21.265890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef021a-8e6d-41e3-a32f-24a0c432c755 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:21.276887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1226/2423] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:08:21.281451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:21.292129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e302ae1-c49e-4f60-aefc-e2e6521fc728 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:21.298450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:21.299216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.299629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.299934 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] PATH_INFO: `/v3/projects/10f29fa5f5f74455b6e4f74849fbc06b/users/b2ff27c0b6d2415ab66cf71ba049369f/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.316206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f29fa5f5f74455b6e4f74849fbc06b, user_id=b2ff27c0b6d2415ab66cf71ba049369f, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:21.318892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:21.333946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1198/2424] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:21 2022] PUT /identity/v3/projects/10f29fa5f5f74455b6e4f74849fbc06b/users/b2ff27c0b6d2415ab66cf71ba049369f/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:21.339345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30045dd5-ad93-40af-b393-bdc0cd782b2a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.339660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30045dd5-ad93-40af-b393-bdc0cd782b2a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.339933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30045dd5-ad93-40af-b393-bdc0cd782b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.359880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-30045dd5-ad93-40af-b393-bdc0cd782b2a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:21.361099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-30045dd5-ad93-40af-b393-bdc0cd782b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:21.420882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1227/2425] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:21.430992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:21.432755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:21.490227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98931b25-14ac-4414-965c-25adeda55685 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:21.493843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:21.494467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:21.494744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:21.495012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:21.504237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:21.505671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:21.515466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1199/2426] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:22.268966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ef960-2aa3-4a55-82f9-4f2fb6668588 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:22.269296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ef960-2aa3-4a55-82f9-4f2fb6668588 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:22.269553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ef960-2aa3-4a55-82f9-4f2fb6668588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:22.282950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:22.301122 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0f6ef960-2aa3-4a55-82f9-4f2fb6668588 None None] MFA Rules not processed for user `b2ff27c0b6d2415ab66cf71ba049369f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:22.302368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0f6ef960-2aa3-4a55-82f9-4f2fb6668588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:22.380110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1228/2427] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:22.384921 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:22.395389 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3f6ac-8d24-4508-a422-ad4b0f0a1fd3 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:22.400267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:22.400891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:22.401173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:22.401445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:22.402413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:22.404603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:22.443640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1200/2428] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:22 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:22.447593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ef960-2aa3-4a55-82f9-4f2fb6668588 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:22.461051 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ef960-2aa3-4a55-82f9-4f2fb6668588 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:22.533595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:22.534317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:22.534609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:22.534887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:22.535568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:22.537526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:22.564344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1229/2429] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:22 2022] POST /identity/v3/users => generated 308 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:22.568413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:22.579373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89ddc6-b57a-421d-b4ed-a0a25e09d8e3 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:22.584069 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:22.584679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:22.584962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:22.585237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:22.585941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:22.587652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:22.599150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1201/2430] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:22.603630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:22.614816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d636ee-3a60-4b22-8a08-e08f0abf7da4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:22.620788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:22.621448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:22.621770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:22.622043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] PATH_INFO: `/v3/projects/cb42d182bb0c452a8c14f7cef54e6844/users/d76c5c102406413eb9a6133a611e606f/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:22.637518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb42d182bb0c452a8c14f7cef54e6844, user_id=d76c5c102406413eb9a6133a611e606f, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:22.640260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:22.652997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1230/2431] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:22 2022] PUT /identity/v3/projects/cb42d182bb0c452a8c14f7cef54e6844/users/d76c5c102406413eb9a6133a611e606f/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:22.657673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:22.667395 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef175c5-06ec-4837-8b0d-91a5cdc37305 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:22.707734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:22.726331 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e787408-c299-406e-9ca5-bdf9b5f7939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:22.726937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787408-c299-406e-9ca5-bdf9b5f7939d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:22.727214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787408-c299-406e-9ca5-bdf9b5f7939d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:22.727500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787408-c299-406e-9ca5-bdf9b5f7939d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:22.728171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e787408-c299-406e-9ca5-bdf9b5f7939d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:22.729765 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e787408-c299-406e-9ca5-bdf9b5f7939d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:22.740226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1202/2432] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:22.744526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:22.754692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafd5ae-64d5-4c91-af8b-c735f6a6c827 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:22.759295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:22.759963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:22.760243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:22.760512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] PATH_INFO: `/v3/projects/cb42d182bb0c452a8c14f7cef54e6844/users/d76c5c102406413eb9a6133a611e606f/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:22.762368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb42d182bb0c452a8c14f7cef54e6844, user_id=d76c5c102406413eb9a6133a611e606f, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:22.765059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:22.774889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1231/2433] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:22 2022] PUT /identity/v3/projects/cb42d182bb0c452a8c14f7cef54e6844/users/d76c5c102406413eb9a6133a611e606f/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:23.535329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113231e-f4a2-4a0c-87df-d06b62ef8047 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:23.535864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113231e-f4a2-4a0c-87df-d06b62ef8047 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:23.535963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113231e-f4a2-4a0c-87df-d06b62ef8047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:23.567740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8113231e-f4a2-4a0c-87df-d06b62ef8047 None None] MFA Rules not processed for user `d76c5c102406413eb9a6133a611e606f`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:23.569471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8113231e-f4a2-4a0c-87df-d06b62ef8047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:23.665856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1203/2434] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:25.353075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:25.355065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:25.366655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113231e-f4a2-4a0c-87df-d06b62ef8047 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:25.426930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:25.449709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113231e-f4a2-4a0c-87df-d06b62ef8047 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:25.452294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1958292-9074-424a-affe-9132ba99421f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:25.454286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:25.454920 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:25.455198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:25.455470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] PATH_INFO: `/v3/users/b2ff27c0b6d2415ab66cf71ba049369f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:25.456599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ff27c0b6d2415ab66cf71ba049369f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:25.457781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:25.458415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:25.458521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:25.458694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:25.458963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:25.460688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:25.473472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:25.475084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:25.487244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1232/2435] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:08:25.512154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ff27c0b6d2415ab66cf71ba049369f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.512818 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ff27c0b6d2415ab66cf71ba049369f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.520032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ff27c0b6d2415ab66cf71ba049369f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.527959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:25.534515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ff27c0b6d2415ab66cf71ba049369f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.535407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ff27c0b6d2415ab66cf71ba049369f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.548392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1204/2436] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:25 2022] DELETE /identity/v3/users/b2ff27c0b6d2415ab66cf71ba049369f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:25.732076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:25.756535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6829f-7b33-4046-a78b-2341d32c2e2e service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:25.821064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:25.821770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:25.822055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:25.822330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] PATH_INFO: `/v3/projects/10f29fa5f5f74455b6e4f74849fbc06b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:25.823954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f29fa5f5f74455b6e4f74849fbc06b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:25.826805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:25.888513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f29fa5f5f74455b6e4f74849fbc06b', 'request_id': 'req-052d155d-242b-4b54-b271-2154f2999763'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.895296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f29fa5f5f74455b6e4f74849fbc06b', 'request_id': 'req-052d155d-242b-4b54-b271-2154f2999763'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.901248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f29fa5f5f74455b6e4f74849fbc06b', 'request_id': 'req-052d155d-242b-4b54-b271-2154f2999763'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.914091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] The token cache is being invalidate because project 10f29fa5f5f74455b6e4f74849fbc06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:25.914439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.915124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:25.916698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1233/2437] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:25 2022] DELETE /identity/v3/projects/10f29fa5f5f74455b6e4f74849fbc06b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:25.921596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:25.923151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:25.936550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897125-8a33-4c74-b50c-77778f0e5a9b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.010917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.012485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.012767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.013041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] PATH_INFO: `/v3/users/d76c5c102406413eb9a6133a611e606f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.014791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76c5c102406413eb9a6133a611e606f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:26.016991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:26.071918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76c5c102406413eb9a6133a611e606f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.072514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76c5c102406413eb9a6133a611e606f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.082164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76c5c102406413eb9a6133a611e606f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.096616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76c5c102406413eb9a6133a611e606f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.097568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da7f0916-8376-498d-bebd-8b677d38efc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76c5c102406413eb9a6133a611e606f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.112269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1205/2438] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:25 2022] DELETE /identity/v3/users/d76c5c102406413eb9a6133a611e606f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:26.291674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:26.293404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:26.309390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d155d-242b-4b54-b271-2154f2999763 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.381384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.382201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.382489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.383146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] PATH_INFO: `/v3/projects/cb42d182bb0c452a8c14f7cef54e6844` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.384002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb42d182bb0c452a8c14f7cef54e6844)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:26.386020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:26.392951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:26.438285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb42d182bb0c452a8c14f7cef54e6844', 'request_id': 'req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.445270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb42d182bb0c452a8c14f7cef54e6844', 'request_id': 'req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.458394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb42d182bb0c452a8c14f7cef54e6844', 'request_id': 'req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.472616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] The token cache is being invalidate because project cb42d182bb0c452a8c14f7cef54e6844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:26.472616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.472994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fe0be6d3-df7c-4223-8aa1-25385ed6caf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:26.474779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1234/2439] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:26 2022] DELETE /identity/v3/projects/cb42d182bb0c452a8c14f7cef54e6844 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:26.485043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77722-f3b6-4ab0-98cd-3312988f5cb9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.485670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77722-f3b6-4ab0-98cd-3312988f5cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.485670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77722-f3b6-4ab0-98cd-3312988f5cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.513013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-03d77722-f3b6-4ab0-98cd-3312988f5cb9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:26.514306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-03d77722-f3b6-4ab0-98cd-3312988f5cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:26.576256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1206/2440] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:26.584153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f356bd-8fec-43ff-831c-237ae57b5f49 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.584444 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f356bd-8fec-43ff-831c-237ae57b5f49 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.584718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f356bd-8fec-43ff-831c-237ae57b5f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.604953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-38f356bd-8fec-43ff-831c-237ae57b5f49 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:26.606160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38f356bd-8fec-43ff-831c-237ae57b5f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:26.615056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1235/2441] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:26.619362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77722-f3b6-4ab0-98cd-3312988f5cb9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:26.624209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:26.630161 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77722-f3b6-4ab0-98cd-3312988f5cb9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.636145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.636747 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.637021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.637288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.637924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:08:26.638219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:26.639892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:26.651759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1207/2442] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:08:26.656745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f356bd-8fec-43ff-831c-237ae57b5f49 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:26.666022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f356bd-8fec-43ff-831c-237ae57b5f49 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.671904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.672050 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.672236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.672501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.673159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:26.675053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:26.710362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1236/2443] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:26 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:26.711741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:26.738457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8528d5-ad48-4273-9444-83b00c048f54 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.821404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.822039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.822379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.822660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.823303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:26.825536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:26.850771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1208/2444] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:26 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:26.854985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:26.883917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b0214-a995-469f-921a-c26cc9bffc6f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.897941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.898868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.899324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.899783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.900647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:26.903372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:26.927448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:26.932869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1237/2445] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:26.938158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed496f-17f6-4c81-a0c3-5d704f14d728 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.943410 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a17d4-f302-46f4-9ac2-acbec4a1c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.944160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a17d4-f302-46f4-9ac2-acbec4a1c3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.944513 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a17d4-f302-46f4-9ac2-acbec4a1c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.944858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a17d4-f302-46f4-9ac2-acbec4a1c3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:26.945583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a17d4-f302-46f4-9ac2-acbec4a1c3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:26.948022 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a17d4-f302-46f4-9ac2-acbec4a1c3a5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:26.967622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1209/2446] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:08:26.967860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:26.978813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842fb31-b2a5-4fdf-bb6b-8b3c5701ac1b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:26.985464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:26.986318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:26.986726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:26.987076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] PATH_INFO: `/v3/projects/a42422dc501f40b09a3a2ee173b6a0a7/users/17e8c00c88fc4525a45678dd754b2202/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.006983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42422dc501f40b09a3a2ee173b6a0a7, user_id=17e8c00c88fc4525a45678dd754b2202, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.012262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.035794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1238/2447] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:26 2022] PUT /identity/v3/projects/a42422dc501f40b09a3a2ee173b6a0a7/users/17e8c00c88fc4525a45678dd754b2202/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:08:27.046767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b9c03-9d3b-42c9-8a6e-c0e59c28589f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.047456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b9c03-9d3b-42c9-8a6e-c0e59c28589f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.047736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b9c03-9d3b-42c9-8a6e-c0e59c28589f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.087245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-733b9c03-9d3b-42c9-8a6e-c0e59c28589f None None] MFA Rules not processed for user `17e8c00c88fc4525a45678dd754b2202`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:27.089041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-733b9c03-9d3b-42c9-8a6e-c0e59c28589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:27.169943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1210/2448] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:27.173349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:27.241422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53d45-0fc1-45a5-ba94-49ff7b6c7397 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:27.249235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:27.249846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.250126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.250393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.251025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.252993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.282361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1239/2449] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:27 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:27.287882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b9c03-9d3b-42c9-8a6e-c0e59c28589f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:27.299443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b9c03-9d3b-42c9-8a6e-c0e59c28589f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:27.357094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:27.363478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:27.364062 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.364680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.365205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.366182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.368289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.391153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1211/2450] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:27 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:27.395067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:27.406994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597906-d817-4d6e-9bae-ffbc2ec28fd7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:27.412494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:27.413605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.413989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.414365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.415116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.416953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.433972 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1240/2451] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:08:27.435258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:27.445726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddbc3d-bdbf-4e95-af8c-b414dbf3314f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:27.450381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:27.451770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.451997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.452267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] PATH_INFO: `/v3/projects/db2633d7f5a244bea758adfabe312c37/users/8f9a74c4d3c54606b0b293fe90cd2925/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.468535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2633d7f5a244bea758adfabe312c37, user_id=8f9a74c4d3c54606b0b293fe90cd2925, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.471227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.502097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1212/2452] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:27 2022] PUT /identity/v3/projects/db2633d7f5a244bea758adfabe312c37/users/8f9a74c4d3c54606b0b293fe90cd2925/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:27.506179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:27.515967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:27.528104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083957b7-10c7-4ed1-919c-774e4848f3f0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:27.627163 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d64e69-137f-4519-b79d-c8b4154a4a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:27.627838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64e69-137f-4519-b79d-c8b4154a4a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.628145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64e69-137f-4519-b79d-c8b4154a4a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.628388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64e69-137f-4519-b79d-c8b4154a4a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.629038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d64e69-137f-4519-b79d-c8b4154a4a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.630709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d64e69-137f-4519-b79d-c8b4154a4a56 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.647534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1241/2453] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:27.650603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:27.668239 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca083dd-5b6c-49f9-be8d-8b8219350ade admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:27.673505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:27.674411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.674791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.675133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] PATH_INFO: `/v3/projects/db2633d7f5a244bea758adfabe312c37/users/8f9a74c4d3c54606b0b293fe90cd2925/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.677283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2633d7f5a244bea758adfabe312c37, user_id=8f9a74c4d3c54606b0b293fe90cd2925, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.680268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.691166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1213/2454] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:27 2022] PUT /identity/v3/projects/db2633d7f5a244bea758adfabe312c37/users/8f9a74c4d3c54606b0b293fe90cd2925/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:08:27.703650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0646aee-029d-4264-ad0a-9155fafc94c4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.704009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0646aee-029d-4264-ad0a-9155fafc94c4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.704353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0646aee-029d-4264-ad0a-9155fafc94c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.739356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d0646aee-029d-4264-ad0a-9155fafc94c4 None None] MFA Rules not processed for user `8f9a74c4d3c54606b0b293fe90cd2925`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:27.740655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d0646aee-029d-4264-ad0a-9155fafc94c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:27.812661 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1242/2455] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:27.823836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:27.825540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:27.884797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a2405-6bea-4b3b-841f-0036cb88cf5c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:27.889786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b03d1-95cd-4510-8098-dc7ee92725ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:27.890408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b03d1-95cd-4510-8098-dc7ee92725ee service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:27.890691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b03d1-95cd-4510-8098-dc7ee92725ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:27.890961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b03d1-95cd-4510-8098-dc7ee92725ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:27.903936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b03d1-95cd-4510-8098-dc7ee92725ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:27.906075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b03d1-95cd-4510-8098-dc7ee92725ee service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:27.915958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1214/2456] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:28.326819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0646aee-029d-4264-ad0a-9155fafc94c4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:28.391632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0646aee-029d-4264-ad0a-9155fafc94c4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:28.396322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:28.396942 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:28.397224 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:28.397618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] PATH_INFO: `/v3/users/17e8c00c88fc4525a45678dd754b2202` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:28.398762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e8c00c88fc4525a45678dd754b2202)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:28.400754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:28.429888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:28.448131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e8c00c88fc4525a45678dd754b2202'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:28.455503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e8c00c88fc4525a45678dd754b2202'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:28.468167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e8c00c88fc4525a45678dd754b2202'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:28.468875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e8c00c88fc4525a45678dd754b2202'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:28.469485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e8c00c88fc4525a45678dd754b2202'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:28.483974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1243/2457] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:28 2022] DELETE /identity/v3/users/17e8c00c88fc4525a45678dd754b2202 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:28.488323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:28.488667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:28.489006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:28.508640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:28.509855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:28.570565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1215/2458] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:08:28.580684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:28.582108 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:28.642158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cff43-1f4e-40d5-9398-be76131adafc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:28.647411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:28.648165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:28.648510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:28.648848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:28.661048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:28.662610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:28.685354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1244/2459] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:29.174878 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:29.176565 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:29.198293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb980-2817-49b9-82e1-789e04c985c8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:29.208213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:29.209771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:29.210480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:29.211764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] PATH_INFO: `/v3/projects/a42422dc501f40b09a3a2ee173b6a0a7` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:29.219362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42422dc501f40b09a3a2ee173b6a0a7)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:29.222975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:29.290167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:29.292704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42422dc501f40b09a3a2ee173b6a0a7', 'request_id': 'req-8ecb9185-67f7-4a14-9696-c15107e58166'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.303430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42422dc501f40b09a3a2ee173b6a0a7', 'request_id': 'req-8ecb9185-67f7-4a14-9696-c15107e58166'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.326612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42422dc501f40b09a3a2ee173b6a0a7', 'request_id': 'req-8ecb9185-67f7-4a14-9696-c15107e58166'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.348782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] The token cache is being invalidate because project a42422dc501f40b09a3a2ee173b6a0a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:29.349077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.349738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.356325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1216/2460] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:29 2022] DELETE /identity/v3/projects/a42422dc501f40b09a3a2ee173b6a0a7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:29.358219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:29.371137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e19707-d6f6-40c3-bdce-bc6dc14ba3f4 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:29.480480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:29.481449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:29.481994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:29.483044 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] PATH_INFO: `/v3/users/8f9a74c4d3c54606b0b293fe90cd2925` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:29.485726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f9a74c4d3c54606b0b293fe90cd2925)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:29.489255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:29.558000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f9a74c4d3c54606b0b293fe90cd2925'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.561418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:29.569373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f9a74c4d3c54606b0b293fe90cd2925'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.586180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f9a74c4d3c54606b0b293fe90cd2925'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.587260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f9a74c4d3c54606b0b293fe90cd2925'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.588324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bbe6ee69-90c9-4997-81a5-4a19afdcc3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f9a74c4d3c54606b0b293fe90cd2925'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:29.606384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1245/2461] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:29 2022] DELETE /identity/v3/users/8f9a74c4d3c54606b0b293fe90cd2925 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:08:30.205995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:30.208132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:30.222581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb9185-67f7-4a14-9696-c15107e58166 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:30.293549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:30.294233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.294515 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.295520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] PATH_INFO: `/v3/projects/db2633d7f5a244bea758adfabe312c37` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.296855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2633d7f5a244bea758adfabe312c37)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:30.298766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:30.359680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2633d7f5a244bea758adfabe312c37', 'request_id': 'req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:30.373736 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2633d7f5a244bea758adfabe312c37', 'request_id': 'req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:30.386322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2633d7f5a244bea758adfabe312c37', 'request_id': 'req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:30.401933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] The token cache is being invalidate because project db2633d7f5a244bea758adfabe312c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:30.402357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:30.403134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-fc266552-41f8-4d8f-b0cd-6c3a6bacb9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:30.405717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1217/2462] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:30 2022] DELETE /identity/v3/projects/db2633d7f5a244bea758adfabe312c37 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:30.411838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f934387-d1d5-4393-85d4-4c8ef6cb3fcb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.412378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f934387-d1d5-4393-85d4-4c8ef6cb3fcb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.412754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f934387-d1d5-4393-85d4-4c8ef6cb3fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.436707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7f934387-d1d5-4393-85d4-4c8ef6cb3fcb None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:30.438250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f934387-d1d5-4393-85d4-4c8ef6cb3fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:30.503198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1246/2463] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:30.512259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d921-2388-486b-80d1-553d69742fa6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.512654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d921-2388-486b-80d1-553d69742fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.513103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d921-2388-486b-80d1-553d69742fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.533688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-79e9d921-2388-486b-80d1-553d69742fa6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:30.535081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-79e9d921-2388-486b-80d1-553d69742fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:30.543379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1218/2464] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:30.547822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f934387-d1d5-4393-85d4-4c8ef6cb3fcb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:30.558523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f934387-d1d5-4393-85d4-4c8ef6cb3fcb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:30.564165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:30.564870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.565211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.565700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.566554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:08:30.566856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:30.568514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:30.580807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1247/2465] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:08:30.585261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d921-2388-486b-80d1-553d69742fa6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:30.597373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d921-2388-486b-80d1-553d69742fa6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:30.609989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:30.610760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.611037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.611311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.611977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:30.613894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:30.657402 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1219/2466] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:30 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:30.658530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:30.669255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f72a50-88fb-4080-bd51-8bae69c27db4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:30.744921 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe094a56-eaca-4654-80a5-a61ccf77e35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:30.745369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094a56-eaca-4654-80a5-a61ccf77e35a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.745721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094a56-eaca-4654-80a5-a61ccf77e35a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.746290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094a56-eaca-4654-80a5-a61ccf77e35a admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.746894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe094a56-eaca-4654-80a5-a61ccf77e35a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:30.749763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe094a56-eaca-4654-80a5-a61ccf77e35a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:30.778627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1248/2467] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:30 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:30.782926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:30.795777 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42165d-2bc8-4f12-bb07-fc18686613af None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.796594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42165d-2bc8-4f12-bb07-fc18686613af None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.797362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42165d-2bc8-4f12-bb07-fc18686613af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.804038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f76c86-a815-48ae-8852-f2a94f530748 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:30.809941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:30.811219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.812000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.812335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.813109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:30.816407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:30.819610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7f42165d-2bc8-4f12-bb07-fc18686613af None None] MFA Rules not processed for user `85cd8a90230b4c69886534be67acd943`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:30.821085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f42165d-2bc8-4f12-bb07-fc18686613af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:30.827998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1220/2468] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:30.832421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:30.843797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2baa-f15a-4bbd-a881-e60c42a87f8c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:30.852440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:30.853046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.853322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.853594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.854259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:30.855880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:30.870654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1221/2469] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:30.872054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:30.883313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c970c39-4e9e-4ea8-9221-be24117366a8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:30.886437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:30.888728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:30.889750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.890237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.890604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] PATH_INFO: `/v3/projects/b776ab9e111e40c0b5a159051fac4ae0/users/0bcdd081f0e04214bdefaa30cbe4f558/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.906684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:30.908905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1249/2470] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:08:30.910413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b776ab9e111e40c0b5a159051fac4ae0, user_id=0bcdd081f0e04214bdefaa30cbe4f558, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:30.913549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:30.935082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1222/2471] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:30 2022] PUT /identity/v3/projects/b776ab9e111e40c0b5a159051fac4ae0/users/0bcdd081f0e04214bdefaa30cbe4f558/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:30.946389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93d0d0-4c86-49c0-964d-01478269b10e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:30.946672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93d0d0-4c86-49c0-964d-01478269b10e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:30.946947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93d0d0-4c86-49c0-964d-01478269b10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:30.985590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2a93d0d0-4c86-49c0-964d-01478269b10e None None] MFA Rules not processed for user `0bcdd081f0e04214bdefaa30cbe4f558`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:30.987356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2a93d0d0-4c86-49c0-964d-01478269b10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:31.069619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1250/2472] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:31.073422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:31.146252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83263f-c04b-43ed-b3b2-587b294369c9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:31.151128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:31.151779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.152189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.152470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.153122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:31.155753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:31.191583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93d0d0-4c86-49c0-964d-01478269b10e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:31.193213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1223/2473] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:31 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:31.204260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93d0d0-4c86-49c0-964d-01478269b10e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:31.294279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:31.295700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.295700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.295700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.296337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:31.298425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:31.326717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1251/2474] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:31 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:31.330313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:31.342758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab462bb-83cd-49d2-8f9f-5a7d4fe82195 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:31.349429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:31.350457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.350806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.351151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.351923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:31.353610 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:31.370744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1224/2475] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:31.372061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:31.385635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100f4b4-5822-4ae9-8db1-393540d55077 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:31.390933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:31.391707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.392093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.392442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] PATH_INFO: `/v3/projects/13bddae833f2438dacf89616db98822c/users/d28ce7c05c8a42139a9730bee21f3324/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.410416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bddae833f2438dacf89616db98822c, user_id=d28ce7c05c8a42139a9730bee21f3324, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:31.413352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:31.432637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1252/2476] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:31 2022] PUT /identity/v3/projects/13bddae833f2438dacf89616db98822c/users/d28ce7c05c8a42139a9730bee21f3324/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:08:31.437484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:31.448497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d55348-e510-406d-821c-3e57d9aa971d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:31.515936 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb4077-2bbe-46ec-90ae-05c537967b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:31.516558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4077-2bbe-46ec-90ae-05c537967b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.516839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4077-2bbe-46ec-90ae-05c537967b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.517113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4077-2bbe-46ec-90ae-05c537967b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.517757 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb4077-2bbe-46ec-90ae-05c537967b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:31.519580 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb4077-2bbe-46ec-90ae-05c537967b7e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:31.554647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:31.556672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1225/2477] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:31.566827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8776169-59b5-45f5-a25e-ac0e3105fbe5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:31.573041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:31.573770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.574124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.574471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] PATH_INFO: `/v3/projects/13bddae833f2438dacf89616db98822c/users/d28ce7c05c8a42139a9730bee21f3324/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.576693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bddae833f2438dacf89616db98822c, user_id=d28ce7c05c8a42139a9730bee21f3324, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:31.579547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:31.597647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1253/2478] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:31 2022] PUT /identity/v3/projects/13bddae833f2438dacf89616db98822c/users/d28ce7c05c8a42139a9730bee21f3324/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:31.608990 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a601a3-be34-4bbd-aecc-6deeebdbd6c0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.609302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a601a3-be34-4bbd-aecc-6deeebdbd6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.609549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a601a3-be34-4bbd-aecc-6deeebdbd6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.640847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-51a601a3-be34-4bbd-aecc-6deeebdbd6c0 None None] MFA Rules not processed for user `d28ce7c05c8a42139a9730bee21f3324`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:31.642134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-51a601a3-be34-4bbd-aecc-6deeebdbd6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:31.710141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:31.757853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1226/2479] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:31.765855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:31.769542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:31.818521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:31.851951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc948d-9b21-48aa-a76d-785347833ab2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:31.872894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa571504-0fa7-42a7-8cbb-ab94d3be9def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:31.873668 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa571504-0fa7-42a7-8cbb-ab94d3be9def service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:31.873946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa571504-0fa7-42a7-8cbb-ab94d3be9def service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:31.874220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa571504-0fa7-42a7-8cbb-ab94d3be9def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:31.894993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa571504-0fa7-42a7-8cbb-ab94d3be9def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:31.899602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa571504-0fa7-42a7-8cbb-ab94d3be9def service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:31.931758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1254/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:32.156130 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a601a3-be34-4bbd-aecc-6deeebdbd6c0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:32.236532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a601a3-be34-4bbd-aecc-6deeebdbd6c0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:32.241912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:32.242906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:32.243168 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:32.243445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] PATH_INFO: `/v3/users/0bcdd081f0e04214bdefaa30cbe4f558` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:32.244732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bcdd081f0e04214bdefaa30cbe4f558)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:32.247437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:32.310036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bcdd081f0e04214bdefaa30cbe4f558'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.310816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bcdd081f0e04214bdefaa30cbe4f558'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.318737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bcdd081f0e04214bdefaa30cbe4f558'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.333570 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bcdd081f0e04214bdefaa30cbe4f558'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.334948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bcdd081f0e04214bdefaa30cbe4f558'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.351788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1227/2481] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:32 2022] DELETE /identity/v3/users/0bcdd081f0e04214bdefaa30cbe4f558 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:08:32.356286 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:32.356566 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:32.356887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:32.378898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:32.380188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:32.444288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1255/2482] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:32.454487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:32.456053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:32.530189 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bfd63-9646-4774-adf2-398737e34250 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:32.533965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:32.534877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:32.535158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:32.535503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:32.545995 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:32.547434 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:32.558011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1228/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:32.847564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:32.849065 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:32.861853 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e66e6c-5b89-4530-afdc-10b237a2176a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:32.867966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:32.868647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:32.868928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:32.869234 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] PATH_INFO: `/v3/projects/b776ab9e111e40c0b5a159051fac4ae0` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:32.870381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b776ab9e111e40c0b5a159051fac4ae0)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:32.872375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:32.944549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b776ab9e111e40c0b5a159051fac4ae0', 'request_id': 'req-eabdeff6-1209-47f6-96e4-43223f4d12d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.951704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b776ab9e111e40c0b5a159051fac4ae0', 'request_id': 'req-eabdeff6-1209-47f6-96e4-43223f4d12d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.959748 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b776ab9e111e40c0b5a159051fac4ae0', 'request_id': 'req-eabdeff6-1209-47f6-96e4-43223f4d12d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.975162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] The token cache is being invalidate because project b776ab9e111e40c0b5a159051fac4ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:32.976285 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.977064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:32.979563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1256/2484] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:32 2022] DELETE /identity/v3/projects/b776ab9e111e40c0b5a159051fac4ae0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:32.983177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:32.994560 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bece5f4-ff8d-46d3-aedf-aa93dfa881c9 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:32.999222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:33.078998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:33.079540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:33.079807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:33.080080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] PATH_INFO: `/v3/users/d28ce7c05c8a42139a9730bee21f3324` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:33.081217 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28ce7c05c8a42139a9730bee21f3324)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:33.083191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:33.155265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28ce7c05c8a42139a9730bee21f3324'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.156047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28ce7c05c8a42139a9730bee21f3324'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.165817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28ce7c05c8a42139a9730bee21f3324'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.184312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28ce7c05c8a42139a9730bee21f3324'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.185714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-688fa73b-9c30-43b8-897e-2d79dfe07422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28ce7c05c8a42139a9730bee21f3324'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.203669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1229/2485] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:32 2022] DELETE /identity/v3/users/d28ce7c05c8a42139a9730bee21f3324 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:33.492914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:33.494766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:33.515413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdeff6-1209-47f6-96e4-43223f4d12d8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:33.594622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:33.601315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:33.601992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:33.602305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:33.602727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] PATH_INFO: `/v3/projects/13bddae833f2438dacf89616db98822c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:33.604144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bddae833f2438dacf89616db98822c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:33.606768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:33.667753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bddae833f2438dacf89616db98822c', 'request_id': 'req-22fdff49-9488-4c64-b613-586b698802b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.674103 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bddae833f2438dacf89616db98822c', 'request_id': 'req-22fdff49-9488-4c64-b613-586b698802b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.680443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bddae833f2438dacf89616db98822c', 'request_id': 'req-22fdff49-9488-4c64-b613-586b698802b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.692982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] The token cache is being invalidate because project 13bddae833f2438dacf89616db98822c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:33.693406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.694162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-22fdff49-9488-4c64-b613-586b698802b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:33.695793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1257/2486] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:33 2022] DELETE /identity/v3/projects/13bddae833f2438dacf89616db98822c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:33.701211 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90029cb7-bba9-43a9-a90c-d67274fcbd32 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:33.701734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90029cb7-bba9-43a9-a90c-d67274fcbd32 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:33.701924 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90029cb7-bba9-43a9-a90c-d67274fcbd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:33.723038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-90029cb7-bba9-43a9-a90c-d67274fcbd32 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:33.724428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-90029cb7-bba9-43a9-a90c-d67274fcbd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:33.786981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1230/2487] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:33.795207 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db934-0f08-4ded-8b61-5aa36a569278 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:33.795515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db934-0f08-4ded-8b61-5aa36a569278 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:33.795797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db934-0f08-4ded-8b61-5aa36a569278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:33.815595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ae0db934-0f08-4ded-8b61-5aa36a569278 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:33.816796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae0db934-0f08-4ded-8b61-5aa36a569278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:33.824694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1258/2488] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:33.829537 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90029cb7-bba9-43a9-a90c-d67274fcbd32 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:33.839514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90029cb7-bba9-43a9-a90c-d67274fcbd32 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:33.845111 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:33.845675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:33.845952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:33.846220 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:33.846856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:08:33.847156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:33.848805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:33.861175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1231/2489] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:08:33.865356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0db934-0f08-4ded-8b61-5aa36a569278 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:33.875751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0db934-0f08-4ded-8b61-5aa36a569278 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:33.880362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:33.880954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:33.881228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:33.881494 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:33.882130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:33.884144 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:33.917271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1259/2490] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:33 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:33.918932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:33.930971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03072cd9-d270-450d-8868-8302aae2d610 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:33.946657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:34.008613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.009320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.009693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.010085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.010794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.012808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.048764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1232/2491] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:33 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:34.052533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.065479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fcf1b-0ee0-465d-a483-4c5105a6cafb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.071185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.072000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.072357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.072694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.073514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.075080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.090459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1260/2492] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:34.092382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.103260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160180-3d8f-42ae-9f2c-301f4fa0a714 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.108833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c50a6-d641-42e7-a645-95e7652a809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.109511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c50a6-d641-42e7-a645-95e7652a809c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.109874 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c50a6-d641-42e7-a645-95e7652a809c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.110336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c50a6-d641-42e7-a645-95e7652a809c admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.111086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c50a6-d641-42e7-a645-95e7652a809c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.112783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c50a6-d641-42e7-a645-95e7652a809c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.124393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1233/2493] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:34.128760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.139694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a361d-7383-42ed-8675-d82f4c07fc98 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.144433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.145077 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.145356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.145625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] PATH_INFO: `/v3/projects/b5d2cfa8d90e41acb043be4f2612bfc5/users/570ca508ac1a4ec5983208dcfae82b95/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.160706 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d2cfa8d90e41acb043be4f2612bfc5, user_id=570ca508ac1a4ec5983208dcfae82b95, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.163344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.179370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1261/2494] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:34 2022] PUT /identity/v3/projects/b5d2cfa8d90e41acb043be4f2612bfc5/users/570ca508ac1a4ec5983208dcfae82b95/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:34.189772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a844c-7004-4ff9-8558-2650b92742db None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.190063 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a844c-7004-4ff9-8558-2650b92742db None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.190339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a844c-7004-4ff9-8558-2650b92742db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.219037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-bc1a844c-7004-4ff9-8558-2650b92742db None None] MFA Rules not processed for user `570ca508ac1a4ec5983208dcfae82b95`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:34.220335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-bc1a844c-7004-4ff9-8558-2650b92742db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:34.288450 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1234/2495] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:34.293601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.360311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801f8d-3c4a-47e1-a12f-b84cd736ba54 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.365188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.365791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.366067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.366336 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.367325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.368901 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.396847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1262/2496] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:34 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:34.401114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a844c-7004-4ff9-8558-2650b92742db None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.412496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a844c-7004-4ff9-8558-2650b92742db None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.471099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.471753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.472037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.472316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.472952 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.474872 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.498643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1235/2497] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:34 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:34.502640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.515803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fca3-2ecc-4a07-96e4-8d1490ba248d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.521120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.521847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.522202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.522553 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.523255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.524966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.527447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:34.540094 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1263/2498] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:34.542961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.553249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd54e9-213d-49d4-be2e-0c397c6b1bfd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.559157 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.560421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.560705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.560981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] PATH_INFO: `/v3/projects/e918748297034adf8cd1ee7670bb6c08/users/235bfe2e009242c2bc957cb678c816fa/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.584039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:34.589665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e918748297034adf8cd1ee7670bb6c08, user_id=235bfe2e009242c2bc957cb678c816fa, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.594370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.608674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1236/2499] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:34 2022] PUT /identity/v3/projects/e918748297034adf8cd1ee7670bb6c08/users/235bfe2e009242c2bc957cb678c816fa/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:34.612871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.624936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67e25a-a46f-4350-b96f-c9bcf8b54f9e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.715165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84619a-bb30-42b8-9a34-2f4a07b21c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.716172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84619a-bb30-42b8-9a34-2f4a07b21c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.716622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84619a-bb30-42b8-9a34-2f4a07b21c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.717057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84619a-bb30-42b8-9a34-2f4a07b21c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.717870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84619a-bb30-42b8-9a34-2f4a07b21c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.719669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84619a-bb30-42b8-9a34-2f4a07b21c3a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.736893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1264/2500] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:34.741449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.754426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2182e-4d42-4685-906f-8a82d35959ac admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:34.760615 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:34.760898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.761171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.761444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] PATH_INFO: `/v3/projects/e918748297034adf8cd1ee7670bb6c08/users/235bfe2e009242c2bc957cb678c816fa/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.765047 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e918748297034adf8cd1ee7670bb6c08, user_id=235bfe2e009242c2bc957cb678c816fa, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:34.768394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:34.796254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1237/2501] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:34 2022] PUT /identity/v3/projects/e918748297034adf8cd1ee7670bb6c08/users/235bfe2e009242c2bc957cb678c816fa/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:34.808803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea86d1a-5b79-43f4-b926-437c3d50d2ff None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:34.809128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea86d1a-5b79-43f4-b926-437c3d50d2ff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:34.809386 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea86d1a-5b79-43f4-b926-437c3d50d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:34.843261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eea86d1a-5b79-43f4-b926-437c3d50d2ff None None] MFA Rules not processed for user `235bfe2e009242c2bc957cb678c816fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:34.846158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eea86d1a-5b79-43f4-b926-437c3d50d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:34.935822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1265/2502] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:34.946787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:34.952440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:35.018448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335eccd-e5a9-408e-9b7b-ef80067e6cb9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:35.022210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f15e5-fe1c-4003-ba7c-5b25e800fe38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:35.022861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f15e5-fe1c-4003-ba7c-5b25e800fe38 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:35.023149 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f15e5-fe1c-4003-ba7c-5b25e800fe38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:35.023598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f15e5-fe1c-4003-ba7c-5b25e800fe38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:35.033738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f15e5-fe1c-4003-ba7c-5b25e800fe38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:35.035185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f15e5-fe1c-4003-ba7c-5b25e800fe38 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:35.052796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1238/2503] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:35.919042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea86d1a-5b79-43f4-b926-437c3d50d2ff None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:35.986633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea86d1a-5b79-43f4-b926-437c3d50d2ff None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:35.991569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:35.992214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:35.992496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:35.992792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] PATH_INFO: `/v3/users/570ca508ac1a4ec5983208dcfae82b95` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:35.993926 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=570ca508ac1a4ec5983208dcfae82b95)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:35.995899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:36.018428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:36.057357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570ca508ac1a4ec5983208dcfae82b95'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.066710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570ca508ac1a4ec5983208dcfae82b95'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.081988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570ca508ac1a4ec5983208dcfae82b95'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.082747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570ca508ac1a4ec5983208dcfae82b95'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.083534 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3057ede0-2d88-46b2-a994-b9f657001b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570ca508ac1a4ec5983208dcfae82b95'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.102183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1266/2504] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:35 2022] DELETE /identity/v3/users/570ca508ac1a4ec5983208dcfae82b95 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:36.119694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:36.121771 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:36.122263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:36.146016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:36.147269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:36.207119 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:36.207424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:36.207722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:36.214304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1239/2505] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:36.226549 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:36.228986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:36.249353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:36.254033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:36.327635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34705-8b93-4c5b-9500-070ecb35e609 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:36.332250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:36.332873 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:36.333156 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:36.333425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:36.341075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1267/2506] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:08:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:08:36.343831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:36.345388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:36.356686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1240/2507] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:08:36.396393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:36.397908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:36.489385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251aabb-1da5-4bb2-b318-b7969b4b3002 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:36.495174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:36.495963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:36.496256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:36.497047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:36.508217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:36.509867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:36.521266 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1268/2508] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:08:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:08:36.708036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:36.725820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:36.769761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7f57b-c888-4caa-abfc-82b047a2f969 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:36.775152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:36.775927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:36.776304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:36.776689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] PATH_INFO: `/v3/projects/b5d2cfa8d90e41acb043be4f2612bfc5` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:36.777999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d2cfa8d90e41acb043be4f2612bfc5)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:36.780645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:36.801203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:36.846165 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d2cfa8d90e41acb043be4f2612bfc5', 'request_id': 'req-ac6cb91b-cc43-4483-a154-8bdb190b9353'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.853535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d2cfa8d90e41acb043be4f2612bfc5', 'request_id': 'req-ac6cb91b-cc43-4483-a154-8bdb190b9353'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.874944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d2cfa8d90e41acb043be4f2612bfc5', 'request_id': 'req-ac6cb91b-cc43-4483-a154-8bdb190b9353'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.894317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] The token cache is being invalidate because project b5d2cfa8d90e41acb043be4f2612bfc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:36.894706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.895303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:36.896971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1241/2509] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:36 2022] DELETE /identity/v3/projects/b5d2cfa8d90e41acb043be4f2612bfc5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:36.900980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:36.912588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34176c-d62c-498f-83fc-4a0c2dd8d2fe service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:36.954772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:36.995848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:36.997199 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:36.998168 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:36.998686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] PATH_INFO: `/v3/users/235bfe2e009242c2bc957cb678c816fa` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:37.000279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235bfe2e009242c2bc957cb678c816fa)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:37.002733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:37.089857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235bfe2e009242c2bc957cb678c816fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.104358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235bfe2e009242c2bc957cb678c816fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.127130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235bfe2e009242c2bc957cb678c816fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.128025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235bfe2e009242c2bc957cb678c816fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.128689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0051e72b-0606-4d6a-ad09-0e83f12cf998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235bfe2e009242c2bc957cb678c816fa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.149169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1269/2510] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:36 2022] DELETE /identity/v3/users/235bfe2e009242c2bc957cb678c816fa => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:08:37.639571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:37.641029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:37.661563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cb91b-cc43-4483-a154-8bdb190b9353 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:37.764667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:37.765805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:37.766278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:37.766937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] PATH_INFO: `/v3/projects/e918748297034adf8cd1ee7670bb6c08` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:37.768341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e918748297034adf8cd1ee7670bb6c08)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:37.773377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:37.890812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e918748297034adf8cd1ee7670bb6c08', 'request_id': 'req-60b9825a-ac0a-454a-af2d-36d5c0485a7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.899262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e918748297034adf8cd1ee7670bb6c08', 'request_id': 'req-60b9825a-ac0a-454a-af2d-36d5c0485a7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.908905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e918748297034adf8cd1ee7670bb6c08', 'request_id': 'req-60b9825a-ac0a-454a-af2d-36d5c0485a7d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.921578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] The token cache is being invalidate because project e918748297034adf8cd1ee7670bb6c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:37.921864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.922707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-60b9825a-ac0a-454a-af2d-36d5c0485a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:37.924671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1242/2511] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:37 2022] DELETE /identity/v3/projects/e918748297034adf8cd1ee7670bb6c08 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:37.929783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab3e74-37a3-4652-b482-9314b19346c5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:37.930142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab3e74-37a3-4652-b482-9314b19346c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:37.930503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab3e74-37a3-4652-b482-9314b19346c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:37.955294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-29ab3e74-37a3-4652-b482-9314b19346c5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:37.956898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-29ab3e74-37a3-4652-b482-9314b19346c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:38.053136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c51e5c-1db4-425b-b6ee-3a2b33d4094e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.053919 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c51e5c-1db4-425b-b6ee-3a2b33d4094e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.054406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c51e5c-1db4-425b-b6ee-3a2b33d4094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.060185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1270/2512] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:38.077498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-68c51e5c-1db4-425b-b6ee-3a2b33d4094e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:38.078992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-68c51e5c-1db4-425b-b6ee-3a2b33d4094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:38.092680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1243/2513] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:08:38.094045 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab3e74-37a3-4652-b482-9314b19346c5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:38.103877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab3e74-37a3-4652-b482-9314b19346c5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:38.111890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:38.112689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.112979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.113255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.113941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:08:38.114447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:38.116473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:38.134593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1271/2514] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:08:38.135798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c51e5c-1db4-425b-b6ee-3a2b33d4094e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:38.149099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c51e5c-1db4-425b-b6ee-3a2b33d4094e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:38.154830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:38.155596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.155882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.156159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.156806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:38.158776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:38.198508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1244/2515] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:38 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:38.203084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:38.215264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36f9cc-65db-4c9b-b131-2796fe081e7e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:38.263884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:38.295390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:38.295983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.296375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.296723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.297788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:38.300175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:38.327138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1272/2516] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:38 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:38.330944 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:38.342794 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d261-7bf2-4086-88c1-254acc916cd8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:38.351782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:38.352931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.353228 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.353629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.355120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:38.359208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:38.359422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:38.371825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1245/2517] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:38.376210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:38.393251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd9e77-4af1-48c5-8851-dae91e7062b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:38.411412 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a5c39-9dea-4798-9c71-dd8b4006c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:38.412078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5c39-9dea-4798-9c71-dd8b4006c3db admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.412361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5c39-9dea-4798-9c71-dd8b4006c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.413176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5c39-9dea-4798-9c71-dd8b4006c3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.413297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a5c39-9dea-4798-9c71-dd8b4006c3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:38.415123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a5c39-9dea-4798-9c71-dd8b4006c3db admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:38.427772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1273/2518] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:38.432056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:38.451176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0438b9-6d6b-4b91-a2af-e5b4a516fa2e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:38.456215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:38.456962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.457315 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.457646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] PATH_INFO: `/v3/projects/4ddbdda5cced48cfb8500b085ae88f12/users/4a6d4cddc3a647f59b1e0d7fd515d563/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.482399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddbdda5cced48cfb8500b085ae88f12, user_id=4a6d4cddc3a647f59b1e0d7fd515d563, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:38.484985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:38.515222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1246/2519] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:38 2022] PUT /identity/v3/projects/4ddbdda5cced48cfb8500b085ae88f12/users/4a6d4cddc3a647f59b1e0d7fd515d563/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:08:38.526356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.526813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.527101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.580237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] MFA Rules not processed for user `4a6d4cddc3a647f59b1e0d7fd515d563`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:38.580237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:38.683134 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1274/2520] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:38.695943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:38.697679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:38.785782 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c136da-ecd0-4e32-97ca-ab587ce8abfe admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:38.791548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba75bce-0954-4e41-ace0-ef8c6c0ee2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:38.792187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba75bce-0954-4e41-ace0-ef8c6c0ee2ed service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:38.792472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba75bce-0954-4e41-ace0-ef8c6c0ee2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:38.793137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba75bce-0954-4e41-ace0-ef8c6c0ee2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:38.803470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba75bce-0954-4e41-ace0-ef8c6c0ee2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:38.805181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba75bce-0954-4e41-ace0-ef8c6c0ee2ed service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:38.824268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1247/2521] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:08:48.676372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:48.819162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:56.063760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:56.065275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:56.124568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b45f4-d4ce-46c5-847e-684a0dcb8326 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:56.128873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dc0c3-fd6a-4a93-8ae6-500626784cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:56.129736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dc0c3-fd6a-4a93-8ae6-500626784cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.130015 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dc0c3-fd6a-4a93-8ae6-500626784cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.130282 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dc0c3-fd6a-4a93-8ae6-500626784cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.132131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d60dc0c3-fd6a-4a93-8ae6-500626784cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:56.142756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dc0c3-fd6a-4a93-8ae6-500626784cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:56.144271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dc0c3-fd6a-4a93-8ae6-500626784cd7 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:56.152064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1275/2522] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:56.275229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86a765-3834-42ae-aaec-9773535e5b36 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.275844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86a765-3834-42ae-aaec-9773535e5b36 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.275844 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86a765-3834-42ae-aaec-9773535e5b36 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.277184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1248/2523] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:08:56.282887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab610f6b-48b8-48a4-81ac-83efcbc06144 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.283226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab610f6b-48b8-48a4-81ac-83efcbc06144 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.283588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab610f6b-48b8-48a4-81ac-83efcbc06144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.304157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ab610f6b-48b8-48a4-81ac-83efcbc06144 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:56.305383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab610f6b-48b8-48a4-81ac-83efcbc06144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:56.375536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1276/2524] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:08:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:56.393480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86a765-3834-42ae-aaec-9773535e5b36 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:56.395250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ee86a765-3834-42ae-aaec-9773535e5b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:56.453082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86a765-3834-42ae-aaec-9773535e5b36 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:56.456652 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a2aaa-6014-4ccb-8698-7c0431f07a56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:56.457252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a2aaa-6014-4ccb-8698-7c0431f07a56 service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.457531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a2aaa-6014-4ccb-8698-7c0431f07a56 service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.457803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a2aaa-6014-4ccb-8698-7c0431f07a56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.468140 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a2aaa-6014-4ccb-8698-7c0431f07a56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:56.469547 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a2aaa-6014-4ccb-8698-7c0431f07a56 service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:56.480198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1249/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:56.550036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3bc3d-902f-4b9b-9ccf-176e591202a3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.550329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3bc3d-902f-4b9b-9ccf-176e591202a3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.550599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3bc3d-902f-4b9b-9ccf-176e591202a3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.552005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1277/2526] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:08:56.557090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ec0f-0735-4ebd-a770-4c03833737ed None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.557377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ec0f-0735-4ebd-a770-4c03833737ed None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.557653 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ec0f-0735-4ebd-a770-4c03833737ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.580598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-9b07ec0f-0735-4ebd-a770-4c03833737ed None None] MFA Rules not processed for user `c0e10b1ef7454167bd1c606ef4f70bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:56.581820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9b07ec0f-0735-4ebd-a770-4c03833737ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:56.588258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1250/2527] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:08:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:56.599939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3bc3d-902f-4b9b-9ccf-176e591202a3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:56.612782 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3bc3d-902f-4b9b-9ccf-176e591202a3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:56.616329 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf658f-e487-479a-985b-6320f7b84093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:56.616927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf658f-e487-479a-985b-6320f7b84093 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.617208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf658f-e487-479a-985b-6320f7b84093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.617483 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf658f-e487-479a-985b-6320f7b84093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.625090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf658f-e487-479a-985b-6320f7b84093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:56.626509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf658f-e487-479a-985b-6320f7b84093 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:56.634199 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1278/2528] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:56.717466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47a1ec-794d-484a-bd49-a29114c44a14 None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.717837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47a1ec-794d-484a-bd49-a29114c44a14 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.718175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47a1ec-794d-484a-bd49-a29114c44a14 None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.719930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1251/2529] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 23:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:08:56.726692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f3a5-3f37-4dc2-823c-006702f71332 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.726955 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f3a5-3f37-4dc2-823c-006702f71332 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.727228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f3a5-3f37-4dc2-823c-006702f71332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.747799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bd55f3a5-3f37-4dc2-823c-006702f71332 None None] MFA Rules not processed for user `786e4e7b1c5147e093693767b229e52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:56.749808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bd55f3a5-3f37-4dc2-823c-006702f71332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:56.759194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1279/2530] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 23:08:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:56.769684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47a1ec-794d-484a-bd49-a29114c44a14 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:56.780486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47a1ec-794d-484a-bd49-a29114c44a14 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:56.789993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463902f-2f0a-4fc2-98a6-518c3f1b555f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:56.789993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463902f-2f0a-4fc2-98a6-518c3f1b555f service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:56.789993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463902f-2f0a-4fc2-98a6-518c3f1b555f service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:56.789993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463902f-2f0a-4fc2-98a6-518c3f1b555f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:56.795800 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463902f-2f0a-4fc2-98a6-518c3f1b555f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:56.797196 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463902f-2f0a-4fc2-98a6-518c3f1b555f service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:56.808057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1252/2531] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:58.060978 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55f3a5-3f37-4dc2-823c-006702f71332 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:58.125189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55f3a5-3f37-4dc2-823c-006702f71332 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:58.129962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:58.130700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:58.131055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:58.131330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] PATH_INFO: `/v3/users/4a6d4cddc3a647f59b1e0d7fd515d563` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:58.132531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a6d4cddc3a647f59b1e0d7fd515d563)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:58.134480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:58.151305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:58.184141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a6d4cddc3a647f59b1e0d7fd515d563'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.191327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a6d4cddc3a647f59b1e0d7fd515d563'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.204315 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a6d4cddc3a647f59b1e0d7fd515d563'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.205070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a6d4cddc3a647f59b1e0d7fd515d563'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.205734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a6d4cddc3a647f59b1e0d7fd515d563'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.220967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1280/2532] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:08:58 2022] DELETE /identity/v3/users/4a6d4cddc3a647f59b1e0d7fd515d563 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:58.225612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34c012-9fca-4742-94e0-6b87b390bc25 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:58.226123 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34c012-9fca-4742-94e0-6b87b390bc25 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:58.226312 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34c012-9fca-4742-94e0-6b87b390bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:58.245510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8e34c012-9fca-4742-94e0-6b87b390bc25 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:58.246714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8e34c012-9fca-4742-94e0-6b87b390bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:58.306400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1253/2533] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:08:58.324098 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:58.325552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:58.384717 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d67c6a-6018-485d-bebe-37ac064a2c8e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:58.388283 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:58.389417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:58.389690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:58.389952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:58.399757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:58.401288 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:58.411820 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1281/2534] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:08:58.761245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:58.761643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:58.761981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:58.782607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:58.783998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:58.842598 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1254/2535] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:08:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:58.848842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:58.850544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:58.856522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:58.857979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:58.863832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce3391-eaf9-4a56-b298-d9592d8b3536 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:58.870906 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:58.871665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:58.872059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:58.872405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] PATH_INFO: `/v3/projects/4ddbdda5cced48cfb8500b085ae88f12` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:58.874514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ddbdda5cced48cfb8500b085ae88f12)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:58.876925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:58.901203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:58.927257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddbdda5cced48cfb8500b085ae88f12', 'request_id': 'req-6773aead-c83d-40ab-86a8-9977e1bdb5f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.933909 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddbdda5cced48cfb8500b085ae88f12', 'request_id': 'req-6773aead-c83d-40ab-86a8-9977e1bdb5f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.940020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddbdda5cced48cfb8500b085ae88f12', 'request_id': 'req-6773aead-c83d-40ab-86a8-9977e1bdb5f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.941659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801db307-f683-42c6-a0c0-6fc25f854e87 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:58.953953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] The token cache is being invalidate because project 4ddbdda5cced48cfb8500b085ae88f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:08:58.954239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.954866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6773aead-c83d-40ab-86a8-9977e1bdb5f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:08:58.956390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1282/2536] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:08:58 2022] DELETE /identity/v3/projects/4ddbdda5cced48cfb8500b085ae88f12 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:58.961153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06152d3-6d4e-4aff-a5cf-66c29edb9eca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:58.961499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06152d3-6d4e-4aff-a5cf-66c29edb9eca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:58.961783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06152d3-6d4e-4aff-a5cf-66c29edb9eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:58.981917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f06152d3-6d4e-4aff-a5cf-66c29edb9eca None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:58.984128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f06152d3-6d4e-4aff-a5cf-66c29edb9eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:58.998966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:59.003763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.004384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.004670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.004954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.006734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:59.049579 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1283/2537] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:08:59.058882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3c8c-a10c-4d4e-a1ad-2beac3074ba9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.059169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3c8c-a10c-4d4e-a1ad-2beac3074ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.059442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3c8c-a10c-4d4e-a1ad-2beac3074ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.069714 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.071301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.078916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ce4b3c8c-a10c-4d4e-a1ad-2beac3074ba9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:59.079717 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1255/2538] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:08:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:08:59.084482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ce4b3c8c-a10c-4d4e-a1ad-2beac3074ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:59.092696 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1284/2539] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:08:59.097037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.110875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd915e5-ed8b-4bb7-8649-170616813ed6 service swift] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.118300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.119038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.119413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.119796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.120890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:08:59.121369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.124018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.140769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1256/2540] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:08:59.145086 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3c8c-a10c-4d4e-a1ad-2beac3074ba9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.160459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3c8c-a10c-4d4e-a1ad-2beac3074ba9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.169568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.171976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.172699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.173756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.174962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.180957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.249128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1285/2541] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:59 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:59.253280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.273151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b39b1e-a955-46dc-9776-33337f3745c8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.388865 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.390064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.390564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.391212 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.392176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.394643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.408759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:08:59.429190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1257/2542] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:59 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:59.433527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.444364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4efdd9-6b9a-477a-9785-1511ae6ca1aa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.449459 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.450185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.450555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.451022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.451787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.453560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.465127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1286/2543] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:59.470009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.482986 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b09364-479d-49a6-a3ef-fb320545fc42 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.492287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53fd2e-bd53-4c9e-a586-72f5e3250c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.492913 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fd2e-bd53-4c9e-a586-72f5e3250c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.493208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fd2e-bd53-4c9e-a586-72f5e3250c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.493459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fd2e-bd53-4c9e-a586-72f5e3250c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.495408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53fd2e-bd53-4c9e-a586-72f5e3250c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.495749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53fd2e-bd53-4c9e-a586-72f5e3250c61 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.511963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1258/2544] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:08:59.513226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.523953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cfedf-ac77-4ddd-89a8-c1cdc12b7711 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.529522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.530369 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.530654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.530932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] PATH_INFO: `/v3/projects/0a4c1c9d3cb24278b0ce68028b346e9a/users/824fb1bc0b5b4e40837b11a7af91b281/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.564147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4c1c9d3cb24278b0ce68028b346e9a, user_id=824fb1bc0b5b4e40837b11a7af91b281, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.567337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.582100 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1287/2545] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:08:59 2022] PUT /identity/v3/projects/0a4c1c9d3cb24278b0ce68028b346e9a/users/824fb1bc0b5b4e40837b11a7af91b281/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:08:59.593190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8ecfd-b1f3-4a85-8a1c-64ace248db43 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.593477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8ecfd-b1f3-4a85-8a1c-64ace248db43 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.593749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8ecfd-b1f3-4a85-8a1c-64ace248db43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.633225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6bb8ecfd-b1f3-4a85-8a1c-64ace248db43 None None] MFA Rules not processed for user `824fb1bc0b5b4e40837b11a7af91b281`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:08:59.634532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6bb8ecfd-b1f3-4a85-8a1c-64ace248db43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:08:59.711477 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1259/2546] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:08:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:08:59.717186 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.791576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479570be-1958-4b94-9579-4cc805cdecf5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.796425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.797030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.797311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.797585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.798223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.800165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.843847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1288/2547] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:08:59 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:08:59.847837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8ecfd-b1f3-4a85-8a1c-64ace248db43 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.860902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8ecfd-b1f3-4a85-8a1c-64ace248db43 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.927358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.928017 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.928298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.928563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.929449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.931376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:08:59.958789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1260/2548] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:08:59 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:08:59.963129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:08:59.980222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e634b18-bb19-4bc1-b78e-bbb8c090f837 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:08:59.993081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:08:59.993713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:08:59.993993 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:08:59.994262 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:08:59.994911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:08:59.996695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.009512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1289/2549] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:09:00.014397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.025542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1526cc0-78a8-4c69-8f30-a5c871f130a9 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.031449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555d77c-da27-467d-902e-fac7d3c4a1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.032215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555d77c-da27-467d-902e-fac7d3c4a1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.032692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555d77c-da27-467d-902e-fac7d3c4a1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.033061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555d77c-da27-467d-902e-fac7d3c4a1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.034021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555d77c-da27-467d-902e-fac7d3c4a1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.035843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555d77c-da27-467d-902e-fac7d3c4a1a2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.052242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1261/2550] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:09:00.059880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.086423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a221a-34c2-489b-8987-6acb56a9dddf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.092701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.093374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.093652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.093925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] PATH_INFO: `/v3/projects/8e239f6af830444b92c4ca8e16630500/users/b8abf3db56164d998552819fdaf123b3/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.111031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e239f6af830444b92c4ca8e16630500, user_id=b8abf3db56164d998552819fdaf123b3, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.114018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.131141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1290/2551] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:09:00 2022] PUT /identity/v3/projects/8e239f6af830444b92c4ca8e16630500/users/b8abf3db56164d998552819fdaf123b3/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:00.142564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988af7d-a057-4b92-b0c7-f91c54c43684 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.142852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988af7d-a057-4b92-b0c7-f91c54c43684 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.143128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988af7d-a057-4b92-b0c7-f91c54c43684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.177502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d988af7d-a057-4b92-b0c7-f91c54c43684 None None] MFA Rules not processed for user `b8abf3db56164d998552819fdaf123b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:00.179143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d988af7d-a057-4b92-b0c7-f91c54c43684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:00.215730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:00.252438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1262/2552] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:00.257063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.267683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:00.327395 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ff076-b331-4da1-b9b4-eeda446124a1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.332172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.332776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.333055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.333330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.333975 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.335937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.363434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1291/2553] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:09:00 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:09:00.367708 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988af7d-a057-4b92-b0c7-f91c54c43684 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.382774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988af7d-a057-4b92-b0c7-f91c54c43684 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.454971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.455608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.455890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.456163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.456804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.458716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.492151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1263/2554] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:09:00 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:09:00.493599 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.506230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045446ff-a7db-4de9-b969-a8375ce62f1a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.511039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027678e-7219-4601-9523-21669da8082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.511708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027678e-7219-4601-9523-21669da8082f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.512250 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027678e-7219-4601-9523-21669da8082f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.512530 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027678e-7219-4601-9523-21669da8082f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.513171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027678e-7219-4601-9523-21669da8082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.515400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027678e-7219-4601-9523-21669da8082f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.538840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1292/2555] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:09:00.539057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.544362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58e1b-2ef2-4136-a798-309b2fd4b897 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.549213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.549875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.550160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.550429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] PATH_INFO: `/v3/projects/77b6f2a3d6504d34996014c737c56f36/users/bc49ab596ea8412bb10b9b2cd4a1f393/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.565679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b6f2a3d6504d34996014c737c56f36, user_id=bc49ab596ea8412bb10b9b2cd4a1f393, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.568545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.582191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1264/2556] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:09:00 2022] PUT /identity/v3/projects/77b6f2a3d6504d34996014c737c56f36/users/bc49ab596ea8412bb10b9b2cd4a1f393/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:09:00.586330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027678e-7219-4601-9523-21669da8082f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.598141 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027678e-7219-4601-9523-21669da8082f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.660814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a4da-8916-40c4-a8d6-12896ee445d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.661441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a4da-8916-40c4-a8d6-12896ee445d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.661729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a4da-8916-40c4-a8d6-12896ee445d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.662524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a4da-8916-40c4-a8d6-12896ee445d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.662655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837a4da-8916-40c4-a8d6-12896ee445d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.664309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837a4da-8916-40c4-a8d6-12896ee445d1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.677479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1293/2557] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:09:00.681978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.695646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b02d7e-0041-447c-bf89-0f39c85e8b17 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.700415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.701692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.701975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.702257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] PATH_INFO: `/v3/projects/77b6f2a3d6504d34996014c737c56f36/users/bc49ab596ea8412bb10b9b2cd4a1f393/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.704823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b6f2a3d6504d34996014c737c56f36, user_id=bc49ab596ea8412bb10b9b2cd4a1f393, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:00.707464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:00.735147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1265/2558] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:09:00 2022] PUT /identity/v3/projects/77b6f2a3d6504d34996014c737c56f36/users/bc49ab596ea8412bb10b9b2cd4a1f393/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:00.745380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.745583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.745860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.774559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] MFA Rules not processed for user `bc49ab596ea8412bb10b9b2cd4a1f393`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:00.775834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:00.854933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1294/2559] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:00.863788 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:00.865738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:00.927096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bb8a8-b82d-493e-aa35-cc87643cbbd5 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:00.931195 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:00.931846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:00.932131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:00.932421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:00.979089 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:01.003131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:01.004900 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:01.014187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1266/2560] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:02.709716 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:02.711192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:02.718913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:02.725197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f799bb7-cac6-4cf9-90d6-0b25ec1d0d51 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:02.729868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:02.730732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:02.731110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:02.731533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:02.832038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:02.833628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:02.843544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1295/2561] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:03.403925 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:03.413233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bde850-3a23-4a25-a48f-582ea6f72cbb service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:03.417020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df18f54-709b-437c-b463-f2ee9f5d5d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:03.417656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df18f54-709b-437c-b463-f2ee9f5d5d94 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:03.417931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df18f54-709b-437c-b463-f2ee9f5d5d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:03.418225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df18f54-709b-437c-b463-f2ee9f5d5d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:03.428055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df18f54-709b-437c-b463-f2ee9f5d5d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:03.429734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df18f54-709b-437c-b463-f2ee9f5d5d94 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:03.439912 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1267/2562] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:07.413809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:07.508419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e00cf-b106-4c4f-8fab-2e31b60ad7ac service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:07.513771 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:07.514452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:07.514713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:07.514984 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] PATH_INFO: `/v3/users/824fb1bc0b5b4e40837b11a7af91b281` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:07.516135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=824fb1bc0b5b4e40837b11a7af91b281)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:07.518057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:07.598814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824fb1bc0b5b4e40837b11a7af91b281'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:07.606804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824fb1bc0b5b4e40837b11a7af91b281'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:07.620323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824fb1bc0b5b4e40837b11a7af91b281'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:07.621302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824fb1bc0b5b4e40837b11a7af91b281'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:07.621984 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824fb1bc0b5b4e40837b11a7af91b281'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:07.637811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1296/2563] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:09:07 2022] DELETE /identity/v3/users/824fb1bc0b5b4e40837b11a7af91b281 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:07.642625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:07.643191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:07.643550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:07.686772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:07.688093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:07.765275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1268/2564] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:09:07.775248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:07.776673 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:07.803659 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:07.848646 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639af218-c1df-4a1a-9061-5f0fc65e85a0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:07.852236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:07.852845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:07.853121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:07.853388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:07.863280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:07.864734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:07.874615 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1297/2565] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:08.146494 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:08.148185 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:08.169491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ceaea-fdd2-4a87-8747-1f7e6f5acc2e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:08.174852 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:08.175758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:08.176180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:08.176896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] PATH_INFO: `/v3/projects/0a4c1c9d3cb24278b0ce68028b346e9a` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:08.177721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4c1c9d3cb24278b0ce68028b346e9a)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:08.180139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:08.228856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4c1c9d3cb24278b0ce68028b346e9a', 'request_id': 'req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.236895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4c1c9d3cb24278b0ce68028b346e9a', 'request_id': 'req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.245439 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4c1c9d3cb24278b0ce68028b346e9a', 'request_id': 'req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.257288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] The token cache is being invalidate because project 0a4c1c9d3cb24278b0ce68028b346e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:09:08.257581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.258206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.259735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1269/2566] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:09:08 2022] DELETE /identity/v3/projects/0a4c1c9d3cb24278b0ce68028b346e9a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:08.264236 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:08.273747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccb5ef-1bd6-43f0-a3ed-caa78afedfc5 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:08.333542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:08.334173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:08.334449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:08.334723 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] PATH_INFO: `/v3/users/b8abf3db56164d998552819fdaf123b3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:08.335859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8abf3db56164d998552819fdaf123b3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:08.337759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:08.382855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8abf3db56164d998552819fdaf123b3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.390549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8abf3db56164d998552819fdaf123b3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.402417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8abf3db56164d998552819fdaf123b3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.403089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8abf3db56164d998552819fdaf123b3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.403730 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8abf3db56164d998552819fdaf123b3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.416596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1298/2567] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:09:08 2022] DELETE /identity/v3/users/b8abf3db56164d998552819fdaf123b3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:08.688536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:08.690109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:08.703444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c72a3-9452-4ce3-8f4b-a8f4b6a4a0b6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:08.764370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:08.775034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:08.775906 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:08.776254 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:08.776711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] PATH_INFO: `/v3/projects/8e239f6af830444b92c4ca8e16630500` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:08.777835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e239f6af830444b92c4ca8e16630500)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:08.780463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:08.834330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e239f6af830444b92c4ca8e16630500', 'request_id': 'req-f0839b1c-fd64-4474-9833-e5dfa3f74fce'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.843266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e239f6af830444b92c4ca8e16630500', 'request_id': 'req-f0839b1c-fd64-4474-9833-e5dfa3f74fce'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.852578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e239f6af830444b92c4ca8e16630500', 'request_id': 'req-f0839b1c-fd64-4474-9833-e5dfa3f74fce'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.867139 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] The token cache is being invalidate because project 8e239f6af830444b92c4ca8e16630500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:09:08.867429 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.868783 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:08.870276 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1270/2568] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:09:08 2022] DELETE /identity/v3/projects/8e239f6af830444b92c4ca8e16630500 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:08.874875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:08.876374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:08.889147 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73912e79-e71c-44a9-9e77-c2d27161b59c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:08.931191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:08.949641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:08.950325 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:08.950605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:08.950876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] PATH_INFO: `/v3/users/bc49ab596ea8412bb10b9b2cd4a1f393` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:08.952029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc49ab596ea8412bb10b9b2cd4a1f393)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:08.953964 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.000557 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc49ab596ea8412bb10b9b2cd4a1f393'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.007287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc49ab596ea8412bb10b9b2cd4a1f393'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.018908 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc49ab596ea8412bb10b9b2cd4a1f393'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.019609 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc49ab596ea8412bb10b9b2cd4a1f393'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.020217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b93b1dfa-8daa-480f-8a23-e4ea100431fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc49ab596ea8412bb10b9b2cd4a1f393'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.032719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1299/2569] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:09:08 2022] DELETE /identity/v3/users/bc49ab596ea8412bb10b9b2cd4a1f393 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:09.298059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.299620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:09.312728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0839b1c-fd64-4474-9833-e5dfa3f74fce admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:09.370337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:09.370973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.371255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.371556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] PATH_INFO: `/v3/projects/77b6f2a3d6504d34996014c737c56f36` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.372691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b6f2a3d6504d34996014c737c56f36)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:09.374589 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.421486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b6f2a3d6504d34996014c737c56f36', 'request_id': 'req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.427582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b6f2a3d6504d34996014c737c56f36', 'request_id': 'req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.435091 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b6f2a3d6504d34996014c737c56f36', 'request_id': 'req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.445403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] The token cache is being invalidate because project 77b6f2a3d6504d34996014c737c56f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:09:09.445688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.446427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc3bad61-d46f-4b1b-819c-1f8a8abb72c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:09.447945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1271/2570] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:09:09 2022] DELETE /identity/v3/projects/77b6f2a3d6504d34996014c737c56f36 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:09:09.453082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994e592-ae07-4473-8711-d4c2bcc23634 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.453588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994e592-ae07-4473-8711-d4c2bcc23634 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.453768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994e592-ae07-4473-8711-d4c2bcc23634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.475618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5994e592-ae07-4473-8711-d4c2bcc23634 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:09.476774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5994e592-ae07-4473-8711-d4c2bcc23634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:09.536383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1300/2571] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:09.544299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306bc7c-c76f-48fb-8abc-0c7f1acd1f9d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.545250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306bc7c-c76f-48fb-8abc-0c7f1acd1f9d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.545522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306bc7c-c76f-48fb-8abc-0c7f1acd1f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.565819 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2306bc7c-c76f-48fb-8abc-0c7f1acd1f9d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:09.567021 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2306bc7c-c76f-48fb-8abc-0c7f1acd1f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:09.574806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1272/2572] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:09.579114 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994e592-ae07-4473-8711-d4c2bcc23634 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.589010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994e592-ae07-4473-8711-d4c2bcc23634 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:09.593605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:09.594192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.594467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.594735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.595366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:09:09.595709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:09.597300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.609041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1301/2573] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:09:09.613164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306bc7c-c76f-48fb-8abc-0c7f1acd1f9d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.622202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306bc7c-c76f-48fb-8abc-0c7f1acd1f9d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:09.627419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:09.628037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.628313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.628583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.629209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:09.631076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.635415 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:09.659745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1273/2574] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:09:09 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:09:09.663905 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.674209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13fe7a-2c2f-4d92-ba0a-3c8b67a46833 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:09.718897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:09.732158 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:09.732700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.732974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.733784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.733891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:09.735768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.756699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1302/2575] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:09:09 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:09:09.761115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.771722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4180d0-484e-4d87-bda2-f715ca8f4b08 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:09.776291 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:09.776890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.777171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.777446 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.778082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:09.780311 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.790043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1274/2576] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:09:09.793843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.803762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b132c-fdf9-4f64-bc2e-0d7716d72d5b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:09.808674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e9f28-b0c7-4ae8-a403-d4b89ec10d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:09.809259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e9f28-b0c7-4ae8-a403-d4b89ec10d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.809541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e9f28-b0c7-4ae8-a403-d4b89ec10d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.809831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e9f28-b0c7-4ae8-a403-d4b89ec10d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.810448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e9f28-b0c7-4ae8-a403-d4b89ec10d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:09.812083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e9f28-b0c7-4ae8-a403-d4b89ec10d0e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.822634 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1303/2577] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:09:09.827286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.837726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263855f-9972-486d-8aff-d6ff57b1d658 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:09.842876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:09.843544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.843826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.844099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] PATH_INFO: `/v3/projects/c1938d9075c844b1a1839ab2f151e471/users/b54bb81f4d274eb683f15394e3e5adcd/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.858296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1938d9075c844b1a1839ab2f151e471, user_id=b54bb81f4d274eb683f15394e3e5adcd, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:09.860945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:09.873969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1275/2578] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:09:09 2022] PUT /identity/v3/projects/c1938d9075c844b1a1839ab2f151e471/users/b54bb81f4d274eb683f15394e3e5adcd/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:09:09.884865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6ded2-9d6a-4b26-af7a-0e7a3ac6dd7b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:09.885152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6ded2-9d6a-4b26-af7a-0e7a3ac6dd7b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:09.885425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6ded2-9d6a-4b26-af7a-0e7a3ac6dd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:09.916145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0fa6ded2-9d6a-4b26-af7a-0e7a3ac6dd7b None None] MFA Rules not processed for user `b54bb81f4d274eb683f15394e3e5adcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:09.917404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0fa6ded2-9d6a-4b26-af7a-0e7a3ac6dd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:09.982531 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1304/2579] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:09.993113 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:09.994826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:10.053008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5c728-bdf4-4326-acf0-6ca3a8dedc56 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:10.056918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494a6fc-2cc3-4204-9291-ef03ce04b5b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:10.057531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a6fc-2cc3-4204-9291-ef03ce04b5b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:10.057811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a6fc-2cc3-4204-9291-ef03ce04b5b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:10.058086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a6fc-2cc3-4204-9291-ef03ce04b5b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:10.077487 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494a6fc-2cc3-4204-9291-ef03ce04b5b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:10.079026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494a6fc-2cc3-4204-9291-ef03ce04b5b8 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:10.088345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1276/2580] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:11.447691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:11.448194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:11.448309 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:11.467373 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] MFA Rules not processed for user `b54bb81f4d274eb683f15394e3e5adcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:11.468590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:11.475268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1305/2581] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:16.424536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b3b7-bbd3-4418-8046-333713e1d797 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:16.425065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b3b7-bbd3-4418-8046-333713e1d797 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:16.425208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b3b7-bbd3-4418-8046-333713e1d797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:16.445245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-90c8b3b7-bbd3-4418-8046-333713e1d797 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:16.446453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-90c8b3b7-bbd3-4418-8046-333713e1d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:16.501154 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1277/2582] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 23:09:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:16.513584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:16.515062 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:16.593427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b17c4-fb70-4799-b4eb-06d1db0864be None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:16.599260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:16.600078 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:16.600350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:16.600619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:16.611170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:16.612816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:16.621694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1306/2583] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:09:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:09:26.493860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:26.617966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:28.742715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a035a-b7ec-4b5b-8025-adca04ae5e9d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:28.743141 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a035a-b7ec-4b5b-8025-adca04ae5e9d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:28.743300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a035a-b7ec-4b5b-8025-adca04ae5e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:28.762713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-b62a035a-b7ec-4b5b-8025-adca04ae5e9d None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:28.763947 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b62a035a-b7ec-4b5b-8025-adca04ae5e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:28.771901 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1278/2584] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:09:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:28.783595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:28.793982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e27a82-c4a0-4fc7-9d4f-5847430a4fbf service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:28.799024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:28.799660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:28.799943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:28.800218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:28.808424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:28.809985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:28.818351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1307/2585] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:09:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:09:29.315337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a669-c07e-48fe-bcdb-b3c869e73534 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:29.315667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a669-c07e-48fe-bcdb-b3c869e73534 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:29.316020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a669-c07e-48fe-bcdb-b3c869e73534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:29.336436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5d25a669-c07e-48fe-bcdb-b3c869e73534 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:29.337648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5d25a669-c07e-48fe-bcdb-b3c869e73534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:29.345335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1279/2586] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 23:09:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:29.357368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:29.368524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b018c4-2431-4e48-80e8-b480c218d53c service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:29.373496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:29.374093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:29.374374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:29.374705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:29.383342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:29.385048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:29.393301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1308/2587] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:09:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:09:29.616176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3a877-35c5-40b5-acec-2b818cb86e15 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:29.616463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3a877-35c5-40b5-acec-2b818cb86e15 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:29.616738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3a877-35c5-40b5-acec-2b818cb86e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:29.635632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-ddf3a877-35c5-40b5-acec-2b818cb86e15 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:29.636831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ddf3a877-35c5-40b5-acec-2b818cb86e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:29.643774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1280/2588] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:09:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:29.655355 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:29.666409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd2e9b-79c3-4b85-b94c-dca7b9553b4e service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:29.672060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:29.672655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:29.672932 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:29.673202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:29.681230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:29.682784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:29.690630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1309/2589] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:09:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:09:29.999427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07350e-d61c-41cd-98cb-67eacefb722b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:30.000572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07350e-d61c-41cd-98cb-67eacefb722b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:30.000572 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07350e-d61c-41cd-98cb-67eacefb722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:30.020345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5e07350e-d61c-41cd-98cb-67eacefb722b None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:30.021546 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5e07350e-d61c-41cd-98cb-67eacefb722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:30.028631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1281/2590] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 23:09:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:30.040979 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:30.051981 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31785db3-8bfc-49cf-a790-62af9ea7c347 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:30.057277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b063d5-5dd8-4c29-8b09-63ab80119a2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:30.057863 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b063d5-5dd8-4c29-8b09-63ab80119a2a service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:30.058143 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b063d5-5dd8-4c29-8b09-63ab80119a2a service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:30.058417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b063d5-5dd8-4c29-8b09-63ab80119a2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:30.066220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b063d5-5dd8-4c29-8b09-63ab80119a2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:30.067806 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b063d5-5dd8-4c29-8b09-63ab80119a2a service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:30.075593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1310/2591] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:09:32.147815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07350e-d61c-41cd-98cb-67eacefb722b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:32.209821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07350e-d61c-41cd-98cb-67eacefb722b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:32.214265 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:32.214890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:32.215170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:32.215442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] PATH_INFO: `/v3/users/b54bb81f4d274eb683f15394e3e5adcd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:32.216609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54bb81f4d274eb683f15394e3e5adcd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:32.219177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:32.264057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54bb81f4d274eb683f15394e3e5adcd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.264706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54bb81f4d274eb683f15394e3e5adcd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.272171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54bb81f4d274eb683f15394e3e5adcd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.284502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54bb81f4d274eb683f15394e3e5adcd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.285074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54bb81f4d274eb683f15394e3e5adcd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.297339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1282/2592] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:09:32 2022] DELETE /identity/v3/users/b54bb81f4d274eb683f15394e3e5adcd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:32.302011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:32.302502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:32.302602 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:32.322178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:32.323363 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:32.381198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1311/2593] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:09:32.392060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:32.393456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:32.449675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb465c-5b2e-426b-b755-f73bc9002178 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:32.453884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3e1fc-8023-4db2-b9e8-a76ce50ed5e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:32.454482 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e1fc-8023-4db2-b9e8-a76ce50ed5e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:32.454764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e1fc-8023-4db2-b9e8-a76ce50ed5e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:32.455030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e1fc-8023-4db2-b9e8-a76ce50ed5e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:32.461713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:32.464000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3e1fc-8023-4db2-b9e8-a76ce50ed5e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:32.466164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3e1fc-8023-4db2-b9e8-a76ce50ed5e3 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:32.476741 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1283/2594] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:32.645417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:32.646923 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:32.659781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf526-330a-414e-b763-79aa0e72a619 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:32.664669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:32.665338 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:32.665625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:32.665894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] PATH_INFO: `/v3/projects/c1938d9075c844b1a1839ab2f151e471` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:32.667170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1938d9075c844b1a1839ab2f151e471)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:32.669253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:32.715245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1938d9075c844b1a1839ab2f151e471', 'request_id': 'req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.721444 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1938d9075c844b1a1839ab2f151e471', 'request_id': 'req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.731828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1938d9075c844b1a1839ab2f151e471', 'request_id': 'req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.744365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] The token cache is being invalidate because project c1938d9075c844b1a1839ab2f151e471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:09:32.744683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.745305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:32.746787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1312/2595] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:09:32 2022] DELETE /identity/v3/projects/c1938d9075c844b1a1839ab2f151e471 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:42.473242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:42.741177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:50.282397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfead5c-3f40-4e66-8aac-a3990686dd5d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:50.282876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfead5c-3f40-4e66-8aac-a3990686dd5d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:50.282982 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfead5c-3f40-4e66-8aac-a3990686dd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:50.303440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1dfead5c-3f40-4e66-8aac-a3990686dd5d None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:50.304940 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1dfead5c-3f40-4e66-8aac-a3990686dd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:50.362286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1284/2596] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 23:09:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:50.386573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:50.388219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:50.466072 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dde5ed-9d2f-4bba-835a-b54c293ef7bb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:50.472126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2fa96-46dd-4978-86c5-22f59ef6f440 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:50.472736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fa96-46dd-4978-86c5-22f59ef6f440 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:50.473009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fa96-46dd-4978-86c5-22f59ef6f440 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:50.473280 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fa96-46dd-4978-86c5-22f59ef6f440 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:50.482133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2fa96-46dd-4978-86c5-22f59ef6f440 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:50.483783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2fa96-46dd-4978-86c5-22f59ef6f440 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:50.492348 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1313/2597] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:09:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:09:54.108493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8906e1-b467-4b99-b37c-8a62a82d783c None None] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:54.109979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8906e1-b467-4b99-b37c-8a62a82d783c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:54.112320 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8906e1-b467-4b99-b37c-8a62a82d783c None None] PATH_INFO: `/` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:54.114016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1285/2598] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 9 23:09:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:09:54.120652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:54.121132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:54.121256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:54.142798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] MFA Rules not processed for user `bd9193fd644f4ec7a39d9c6ab8a1e525`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:54.144508 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:54.222040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1314/2599] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 9 23:09:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:54.230413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8906e1-b467-4b99-b37c-8a62a82d783c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:54.245445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8906e1-b467-4b99-b37c-8a62a82d783c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:54.251665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:54.252293 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:54.252578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:54.252853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:54.254746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:54.267970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:54.269859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:54.282654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1286/2600] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:09:55.238644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:55.240188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:55.303983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a0079-4f88-4c8c-ae46-35ff6eeab5c5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:55.309023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:55.309658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:55.309941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:55.310213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:55.313120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:55.374056 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:55.405171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:55.406685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:55.428683 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1315/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:57.498743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:57.540126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920a546-13ae-40b1-b8c5-5d8cbd2dfeb1 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:57.551976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:57.554493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:57.566010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:57.566644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:57.566927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:57.567203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:57.572552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb95e6-f8b7-4a54-a9f3-5d179d5ca0bd service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:57.580401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:57.586751 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:57.588155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:57.589467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:57.589998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] PATH_INFO: `/v3/users/1bdf9e74be004e8ab24b9a15bb8904d2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:57.596277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdf9e74be004e8ab24b9a15bb8904d2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:57.598701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:57.604642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:57.606500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:57.622827 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1287/2602] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:09:57.683828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdf9e74be004e8ab24b9a15bb8904d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:57.692666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdf9e74be004e8ab24b9a15bb8904d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:57.707140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdf9e74be004e8ab24b9a15bb8904d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:57.707992 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdf9e74be004e8ab24b9a15bb8904d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:57.708653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdf9e74be004e8ab24b9a15bb8904d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:57.743674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1316/2603] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:09:57 2022] DELETE /identity/v3/users/1bdf9e74be004e8ab24b9a15bb8904d2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:58.069479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:58.080914 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e25c-884e-446e-9c9a-a296de8bd8a4 service placement] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:58.145386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:58.146112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:58.146448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:58.146789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] PATH_INFO: `/v3/projects/52565b3ddbe9435e8112f134f6c5d496` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:58.148001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52565b3ddbe9435e8112f134f6c5d496)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:58.150138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:58.206467 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52565b3ddbe9435e8112f134f6c5d496', 'request_id': 'req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.212812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52565b3ddbe9435e8112f134f6c5d496', 'request_id': 'req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.220745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52565b3ddbe9435e8112f134f6c5d496', 'request_id': 'req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.231879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] The token cache is being invalidate because project 52565b3ddbe9435e8112f134f6c5d496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:09:58.232171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.232813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.234978 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1288/2604] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:09:58 2022] DELETE /identity/v3/projects/52565b3ddbe9435e8112f134f6c5d496 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:09:58.239089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:58.240701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:58.254755 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e43b76-3a22-4fcf-8563-86acba0f1d4f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:58.325770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:58.326742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:58.327091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:58.328106 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] PATH_INFO: `/v3/users/11a237c3d68c4634bfc8f56b01361582` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:58.328780 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a237c3d68c4634bfc8f56b01361582)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:58.331571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:58.402773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a237c3d68c4634bfc8f56b01361582'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.405061 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:58.415541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a237c3d68c4634bfc8f56b01361582'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.431271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a237c3d68c4634bfc8f56b01361582'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.432299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a237c3d68c4634bfc8f56b01361582'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.433066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fbc2efac-f05d-48ff-9ecf-4ccfbdb0e433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a237c3d68c4634bfc8f56b01361582'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.459733 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1317/2605] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:09:58 2022] DELETE /identity/v3/users/11a237c3d68c4634bfc8f56b01361582 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:58.654551 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:58.656976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:58.677898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e046b23-4c5d-4eec-a6c1-bd071d9ffab4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:58.693564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:58.756953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:58.758158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:58.758758 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:58.759332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] PATH_INFO: `/v3/projects/16bf50d1c8264278bbe680d5096d3ce0` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:58.762240 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16bf50d1c8264278bbe680d5096d3ce0)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:58.766495 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:58.845199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16bf50d1c8264278bbe680d5096d3ce0', 'request_id': 'req-e684c707-226e-4f81-847a-eb3711363356'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.853862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16bf50d1c8264278bbe680d5096d3ce0', 'request_id': 'req-e684c707-226e-4f81-847a-eb3711363356'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.872739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16bf50d1c8264278bbe680d5096d3ce0', 'request_id': 'req-e684c707-226e-4f81-847a-eb3711363356'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.894640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] The token cache is being invalidate because project 16bf50d1c8264278bbe680d5096d3ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:09:58.895544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.896179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e684c707-226e-4f81-847a-eb3711363356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:58.898338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1289/2606] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:09:58 2022] DELETE /identity/v3/projects/16bf50d1c8264278bbe680d5096d3ce0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:58.903944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529006e-c492-46b7-ad65-18ff14bfab2b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:58.904449 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529006e-c492-46b7-ad65-18ff14bfab2b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:58.904547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529006e-c492-46b7-ad65-18ff14bfab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:58.934610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9529006e-c492-46b7-ad65-18ff14bfab2b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:58.936124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9529006e-c492-46b7-ad65-18ff14bfab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:59.023741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1318/2607] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:59.032676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fb3e8-e3d9-443a-8f6b-2449921a3428 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.033587 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fb3e8-e3d9-443a-8f6b-2449921a3428 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.034081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fb3e8-e3d9-443a-8f6b-2449921a3428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.077604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-036fb3e8-e3d9-443a-8f6b-2449921a3428 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:59.080102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-036fb3e8-e3d9-443a-8f6b-2449921a3428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:59.091843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1290/2608] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:09:59.096785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529006e-c492-46b7-ad65-18ff14bfab2b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.114068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529006e-c492-46b7-ad65-18ff14bfab2b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.119297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.119971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.120544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.120828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.121491 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:09:59.121797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:59.123414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:59.148656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fb3e8-e3d9-443a-8f6b-2449921a3428 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.157699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1319/2609] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:09:59.162721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fb3e8-e3d9-443a-8f6b-2449921a3428 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.168577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.169289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.170307 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.171086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.172918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:59.177485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:59.237699 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.251666 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1291/2610] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:09:59 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:09:59.252802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eb45e-7b3b-4af2-b2b1-570d581f46b8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.323149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.323830 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.324112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.324388 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.325354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:59.326976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:59.390856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1320/2611] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:09:59 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:09:59.392613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.407170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b663a-74e0-4b33-b9ed-6aad4168bb35 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.421975 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.422895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.423344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.423806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.424609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:59.426694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:59.461261 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1292/2612] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:09:59.465253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.481478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.482999 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171bf14-0ca1-4e79-b9bb-fbbaa285698b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.485578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:59.489891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.490776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.491287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.491787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.493127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:59.495167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:59.513946 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb054-d583-4363-b19f-691d7bb4a888 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.518823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1321/2613] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:09:59.520132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.526404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.527388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.527956 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.528420 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] PATH_INFO: `/v3/users/85cd8a90230b4c69886534be67acd943` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.529845 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=85cd8a90230b4c69886534be67acd943)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:59.531861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506a95e-2305-4f31-8b5d-7a3148cb692e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.532603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:59.553888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a0d7c-ccce-4de7-847f-aab4e4284143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.554843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0d7c-ccce-4de7-847f-aab4e4284143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.555110 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0d7c-ccce-4de7-847f-aab4e4284143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.557070 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0d7c-ccce-4de7-847f-aab4e4284143 admin admin] PATH_INFO: `/v3/projects/f68efb39c9f649d6a0e2bc424b418cbd/users/b339e5581cf044d9a55eb85561a58106/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.600403 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a0d7c-ccce-4de7-847f-aab4e4284143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68efb39c9f649d6a0e2bc424b418cbd, user_id=b339e5581cf044d9a55eb85561a58106, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:09:59.603533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a0d7c-ccce-4de7-847f-aab4e4284143 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:09:59.622543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85cd8a90230b4c69886534be67acd943'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:59.623245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85cd8a90230b4c69886534be67acd943'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:59.626956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1322/2614] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:09:59 2022] PUT /identity/v3/projects/f68efb39c9f649d6a0e2bc424b418cbd/users/b339e5581cf044d9a55eb85561a58106/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:59.630255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:59.631867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.632157 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.632430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:09:59.633839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85cd8a90230b4c69886534be67acd943'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:59.653245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85cd8a90230b4c69886534be67acd943'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:59.654314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85cd8a90230b4c69886534be67acd943'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:09:59.656408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:09:59.658351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:59.663275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:09:59.669759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1293/2615] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:09:59 2022] DELETE /identity/v3/users/85cd8a90230b4c69886534be67acd943 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:09:59.803550 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1323/2616] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:09:59.816926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:09:59.819277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:09:59.929559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b5af8-9794-45e9-85f5-846dec05f5bd admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:09:59.933387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9844f2-6ad0-4f1d-8dae-1d27b44c47eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:09:59.934636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9844f2-6ad0-4f1d-8dae-1d27b44c47eb service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:09:59.934636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9844f2-6ad0-4f1d-8dae-1d27b44c47eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:09:59.934977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9844f2-6ad0-4f1d-8dae-1d27b44c47eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.001438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.003059 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:00.023595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69588751-f635-4a9f-8fc5-ad6b56f0f306 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:00.036684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:00.037507 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.038339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.038619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] PATH_INFO: `/v3/projects/cba38c443abf49a48bdb55f42dac68b0` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.040945 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba38c443abf49a48bdb55f42dac68b0)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.042918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.043574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9844f2-6ad0-4f1d-8dae-1d27b44c47eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.045383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9844f2-6ad0-4f1d-8dae-1d27b44c47eb service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.070659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1294/2617] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:00.126941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba38c443abf49a48bdb55f42dac68b0', 'request_id': 'req-947dc96b-2a22-4335-9527-20d3d03f24a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:00.138860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba38c443abf49a48bdb55f42dac68b0', 'request_id': 'req-947dc96b-2a22-4335-9527-20d3d03f24a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:00.146903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba38c443abf49a48bdb55f42dac68b0', 'request_id': 'req-947dc96b-2a22-4335-9527-20d3d03f24a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:00.163801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] The token cache is being invalidate because project cba38c443abf49a48bdb55f42dac68b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:00.164323 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:00.165041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-947dc96b-2a22-4335-9527-20d3d03f24a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:00.166994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1324/2618] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:09:59 2022] DELETE /identity/v3/projects/cba38c443abf49a48bdb55f42dac68b0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:00.172507 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd51fe-bc94-4058-91b4-51b8ec4c24e7 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.172890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd51fe-bc94-4058-91b4-51b8ec4c24e7 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.173232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd51fe-bc94-4058-91b4-51b8ec4c24e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.195713 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-d1fd51fe-bc94-4058-91b4-51b8ec4c24e7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:00.197046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-d1fd51fe-bc94-4058-91b4-51b8ec4c24e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:00.275552 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1295/2619] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:00.276590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640e3a-a737-42f0-9d83-a98811fc1ce6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.276882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640e3a-a737-42f0-9d83-a98811fc1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.277160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640e3a-a737-42f0-9d83-a98811fc1ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.301789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4a640e3a-a737-42f0-9d83-a98811fc1ce6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:00.302513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4a640e3a-a737-42f0-9d83-a98811fc1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:00.315513 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1325/2620] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:00.316843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd51fe-bc94-4058-91b4-51b8ec4c24e7 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.326694 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd51fe-bc94-4058-91b4-51b8ec4c24e7 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:00.332133 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:00.332739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.333025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.333301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.333945 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:00.334246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.335889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.349609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1296/2621] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:10:00.354085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640e3a-a737-42f0-9d83-a98811fc1ce6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.364617 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640e3a-a737-42f0-9d83-a98811fc1ce6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:00.369501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:00.370191 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.370484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.370759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.371439 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.373480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.424013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1326/2622] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:00 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:00.427037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.440248 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2daf6b-be00-4c6f-9c9a-9f2bcaee6983 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:00.446690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:00.539548 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:00.541358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.541817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.542306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.543858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.550405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.585012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1297/2623] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:00 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:00.589302 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.607409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44fc85-a9a8-456e-b951-f21a07eb2bbe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:00.616295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:00.616933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.617225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.617501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.618295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.620057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.634458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1327/2624] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:00.638635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.649479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3becb7-86ba-4a3e-9303-206deb519f70 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:00.655386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1ce1b-12dc-44ef-b551-d80d2d03d778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:00.655657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1ce1b-12dc-44ef-b551-d80d2d03d778 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.656449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1ce1b-12dc-44ef-b551-d80d2d03d778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.656712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1ce1b-12dc-44ef-b551-d80d2d03d778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.657359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1ce1b-12dc-44ef-b551-d80d2d03d778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.658960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1ce1b-12dc-44ef-b551-d80d2d03d778 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.670199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1298/2625] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:00.675102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.688226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce261993-1ddc-413e-9f99-2803d1b6a3ba admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:00.694116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:00.695366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.695366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.695633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] PATH_INFO: `/v3/projects/8fbdd29bbd314ac683f35199c123662a/users/8d827112547b4b3095d838430fd58114/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.711556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbdd29bbd314ac683f35199c123662a, user_id=8d827112547b4b3095d838430fd58114, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:00.714402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:00.738344 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1328/2626] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:00 2022] PUT /identity/v3/projects/8fbdd29bbd314ac683f35199c123662a/users/8d827112547b4b3095d838430fd58114/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:00.751778 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0eec-e3bf-429f-bbbe-912ef11f7016 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:00.752273 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0eec-e3bf-429f-bbbe-912ef11f7016 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:00.752555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0eec-e3bf-429f-bbbe-912ef11f7016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:00.794364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-93ad0eec-e3bf-429f-bbbe-912ef11f7016 None None] MFA Rules not processed for user `8d827112547b4b3095d838430fd58114`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:00.795910 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-93ad0eec-e3bf-429f-bbbe-912ef11f7016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:00.886414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1299/2627] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:00.900832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:00.903301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:00.955948 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:01.024907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0775e-c3ac-418c-b325-2b91cbe8d79e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:01.047631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:01.048597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:01.050190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:01.050656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:01.081610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:01.084528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:01.106521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1329/2628] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:10:04.195859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad0eec-e3bf-429f-bbbe-912ef11f7016 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:04.273074 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad0eec-e3bf-429f-bbbe-912ef11f7016 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:04.281208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:04.281958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:04.282362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:04.282727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] PATH_INFO: `/v3/users/8d827112547b4b3095d838430fd58114` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:04.284059 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d827112547b4b3095d838430fd58114)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:04.286083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:04.309860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:04.364529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d827112547b4b3095d838430fd58114'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:04.366044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d827112547b4b3095d838430fd58114'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:04.375651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d827112547b4b3095d838430fd58114'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:04.399651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d827112547b4b3095d838430fd58114'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:04.401327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-4e828daa-74da-4105-a819-92f834c0fe2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d827112547b4b3095d838430fd58114'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:04.425184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:04.426902 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:04.427770 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1300/2629] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:04 2022] DELETE /identity/v3/users/8d827112547b4b3095d838430fd58114 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:04.432180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:04.432755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:04.433144 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:04.458815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:04.460539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:04.520989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44d2dd-f07a-4548-83de-dcc69339404b service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:04.585430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:04.586064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:04.586343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:04.587540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:04.591810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:04.653636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1301/2630] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:04.665660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:04.667341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:04.683294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4255be-5a64-4013-a3ac-fa4cd05c1167 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:04.688383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20eea5-bff7-485d-b031-437a730df953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:04.689353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20eea5-bff7-485d-b031-437a730df953 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:04.689737 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20eea5-bff7-485d-b031-437a730df953 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:04.690187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20eea5-bff7-485d-b031-437a730df953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:04.693324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:04.695426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:04.703438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20eea5-bff7-485d-b031-437a730df953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:04.705096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20eea5-bff7-485d-b031-437a730df953 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:04.706888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1330/2631] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:04.721290 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1302/2632] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:05.011865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.013946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.038198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:05.077297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9771e-cd43-4985-ade6-f2ceb0fb15d6 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.081638 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.082251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.082997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.082997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.085232 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.165177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.166872 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.178182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1331/2633] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:05.291126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.291923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.292359 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.317604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.319194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.343241 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d774ba-4a8b-4757-a913-7cde70b9286b service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.358424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.359246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.359592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.360133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] PATH_INFO: `/v3/projects/8fbdd29bbd314ac683f35199c123662a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.370287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fbdd29bbd314ac683f35199c123662a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.371436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] MFA Rules not processed for user `b339e5581cf044d9a55eb85561a58106`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:05.375469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.377480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.526106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1303/2634] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:05.531332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.533171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.548288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531468c5-04a0-4f18-984b-c480a710bb37 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.553444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.554398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.554593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.555545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.555716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.558353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.571883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:05.605926 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1304/2635] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:05 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:10:05.608198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.621976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fbdd29bbd314ac683f35199c123662a', 'request_id': 'req-6166f269-fcc3-409b-94d0-0856e1ee44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:05.629671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fbdd29bbd314ac683f35199c123662a', 'request_id': 'req-6166f269-fcc3-409b-94d0-0856e1ee44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:05.641760 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fbdd29bbd314ac683f35199c123662a', 'request_id': 'req-6166f269-fcc3-409b-94d0-0856e1ee44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:05.659226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] The token cache is being invalidate because project 8fbdd29bbd314ac683f35199c123662a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:05.659886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:05.661549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6166f269-fcc3-409b-94d0-0856e1ee44ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:05.664263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1332/2636] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:05 2022] DELETE /identity/v3/projects/8fbdd29bbd314ac683f35199c123662a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:05.669713 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0c3c-10c3-4300-804b-362fc21a5727 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.670075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0c3c-10c3-4300-804b-362fc21a5727 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.670416 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0c3c-10c3-4300-804b-362fc21a5727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.694859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db232c0-db7f-4208-bf99-79bfce880a62 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.696487 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-483a0c3c-10c3-4300-804b-362fc21a5727 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:05.698536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-483a0c3c-10c3-4300-804b-362fc21a5727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.775596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:05.789490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.790814 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.792741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1333/2637] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:05.802733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.803326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.803703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e63d2-6899-4eaa-8c24-18dcb3f9c2b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.804542 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e63d2-6899-4eaa-8c24-18dcb3f9c2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.804968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e63d2-6899-4eaa-8c24-18dcb3f9c2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.805468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.808468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.843378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1305/2638] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:05 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:05.849869 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.850422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1d6e63d2-6899-4eaa-8c24-18dcb3f9c2b5 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:05.852924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1d6e63d2-6899-4eaa-8c24-18dcb3f9c2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.855902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:05.874951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1334/2639] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:05.879856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e63d2-6899-4eaa-8c24-18dcb3f9c2b5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.885211 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b298a2-7e26-4c97-9224-8d42b545697d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.892970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e63d2-6899-4eaa-8c24-18dcb3f9c2b5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.895542 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.896295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.896673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.897170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.898019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.899658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.910842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.911940 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.912676 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.913125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.913460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1306/2640] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:05.914273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:05.914596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.916571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.917607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.930719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bca0d-a602-4b90-a3c5-1b7d778396ce admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.937830 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.938733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.939227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.942973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1335/2641] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:10:05.943464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] PATH_INFO: `/v3/projects/811c36b765454ac28ed99a50c3a801c5/users/0752c7b0aa574a6ab87af239daeb178a/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.945053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:05.962209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c11d4-775d-4d75-b4ad-57e594046b7d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:05.966040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] RBAC: Authorizing `identity:create_grant(project_id=811c36b765454ac28ed99a50c3a801c5, user_id=0752c7b0aa574a6ab87af239daeb178a, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.969803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.973524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:05.975392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:05.975759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:05.976162 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:05.977451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:05.979810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:05.995338 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1307/2642] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:05 2022] PUT /identity/v3/projects/811c36b765454ac28ed99a50c3a801c5/users/0752c7b0aa574a6ab87af239daeb178a/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:05.996816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.034102 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1336/2643] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:05 2022] POST /identity/v3/projects => generated 359 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:06.038772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.066898 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:06.082283 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584267d5-45db-4676-83ce-6bf50578c68e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.121682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841166e-af46-4a28-a552-ae59cb140f92 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.127299 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.128000 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.128287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.128559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.129222 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.132600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.156571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1337/2644] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:06 2022] POST /identity/v3/users => generated 280 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:06.160726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.161489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.162167 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.162451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.162722 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.163472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.166191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.173770 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13ab2a-df3d-4f52-8821-11befbf77cf6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.179679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.180098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1308/2645] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:06.180320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.180624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.180895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.181562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.183433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.184369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.198994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1338/2646] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:06.199209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145687df-3fd2-4f14-90bc-d0be53f119bc admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.204088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.204996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.205411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.205817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] PATH_INFO: `/v3/projects/811c36b765454ac28ed99a50c3a801c5/users/0752c7b0aa574a6ab87af239daeb178a/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.206944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.208222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=811c36b765454ac28ed99a50c3a801c5, user_id=0752c7b0aa574a6ab87af239daeb178a, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.211391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.224810 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b6e8-7613-478a-b030-75673552c56d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.230480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1309/2647] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:06 2022] PUT /identity/v3/projects/811c36b765454ac28ed99a50c3a801c5/users/0752c7b0aa574a6ab87af239daeb178a/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:10:06.306994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cec93a-4b4d-488f-86bb-9aacd5914259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.307815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cec93a-4b4d-488f-86bb-9aacd5914259 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.308321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cec93a-4b4d-488f-86bb-9aacd5914259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.308686 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cec93a-4b4d-488f-86bb-9aacd5914259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.309849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cec93a-4b4d-488f-86bb-9aacd5914259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.312510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cec93a-4b4d-488f-86bb-9aacd5914259 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.329201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1339/2648] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:06.333941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.349145 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf5a0e-45b1-49c0-bbcf-17b47e2c1f0d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.358250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.359835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.360132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.360631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] PATH_INFO: `/v3/projects/7cf763f59a3447a497ee0e22e3a1bcca/users/5b887d6c3cf64e80a161eea6ec360ba9/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.386817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf763f59a3447a497ee0e22e3a1bcca, user_id=5b887d6c3cf64e80a161eea6ec360ba9, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.390962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.416514 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1310/2649] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:06 2022] PUT /identity/v3/projects/7cf763f59a3447a497ee0e22e3a1bcca/users/5b887d6c3cf64e80a161eea6ec360ba9/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:10:06.428226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8c9a-ee02-46f8-b3b1-53f25018daca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.429304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8c9a-ee02-46f8-b3b1-53f25018daca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.429304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8c9a-ee02-46f8-b3b1-53f25018daca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.468885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0dde8c9a-ee02-46f8-b3b1-53f25018daca None None] MFA Rules not processed for user `5b887d6c3cf64e80a161eea6ec360ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:06.470505 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0dde8c9a-ee02-46f8-b3b1-53f25018daca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:06.521347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:06.587182 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1340/2650] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:06.592403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.682229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acc5e9-8d61-4c4c-9db3-85e20415a28d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.697476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.701796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.702118 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.702879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.704426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.707833 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.759531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1311/2651] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:06 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:06.762709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde8c9a-ee02-46f8-b3b1-53f25018daca None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.778428 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde8c9a-ee02-46f8-b3b1-53f25018daca None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.875264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.877175 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.878184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.878894 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.880849 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.883583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.923618 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1341/2652] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:06 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:06.927908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:06.942428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7875-2f3d-45b5-97bc-a665e8b98262 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:06.948057 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:06.948908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:06.949411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:06.949887 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:06.950877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:06.953339 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:06.970576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1312/2653] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:06.974705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:07.001721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e25c-c29a-4009-bf59-da992e8c41c5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:07.015801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:07.016729 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:07.017206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:07.017637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] PATH_INFO: `/v3/projects/4224481659d94636b257d09b1860b2c9/users/c338286b8ffc414694cd3a21348cf913/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:07.042007 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4224481659d94636b257d09b1860b2c9, user_id=c338286b8ffc414694cd3a21348cf913, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:07.047591 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:07.071164 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1342/2654] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:06 2022] PUT /identity/v3/projects/4224481659d94636b257d09b1860b2c9/users/c338286b8ffc414694cd3a21348cf913/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:07.073082 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:07.092574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a99e-7bbe-4415-9699-d9226cf550a7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:07.150281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:07.188351 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5348d1-71dc-4eba-9199-d7abf79c4721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:07.189325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5348d1-71dc-4eba-9199-d7abf79c4721 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:07.189969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5348d1-71dc-4eba-9199-d7abf79c4721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:07.190414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5348d1-71dc-4eba-9199-d7abf79c4721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:07.191531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5348d1-71dc-4eba-9199-d7abf79c4721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:07.193553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5348d1-71dc-4eba-9199-d7abf79c4721 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:07.207931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1313/2655] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:07.212036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:07.225841 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed19e8-b56b-40c1-9bb4-f0c01819dc8c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:07.235244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:07.236434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:07.236434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:07.236773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] PATH_INFO: `/v3/projects/4224481659d94636b257d09b1860b2c9/users/c338286b8ffc414694cd3a21348cf913/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:07.239457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4224481659d94636b257d09b1860b2c9, user_id=c338286b8ffc414694cd3a21348cf913, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:07.242827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:07.259952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1343/2656] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:07 2022] PUT /identity/v3/projects/4224481659d94636b257d09b1860b2c9/users/c338286b8ffc414694cd3a21348cf913/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:10:07.276541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c36d7-cfc5-4f62-b60d-5ccdb4ad55bd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:07.276541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c36d7-cfc5-4f62-b60d-5ccdb4ad55bd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:07.277164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c36d7-cfc5-4f62-b60d-5ccdb4ad55bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:07.563207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-878c36d7-cfc5-4f62-b60d-5ccdb4ad55bd None None] MFA Rules not processed for user `c338286b8ffc414694cd3a21348cf913`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:07.565370 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-878c36d7-cfc5-4f62-b60d-5ccdb4ad55bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:07.674104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1314/2657] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:07.681006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:07.683704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:07.819467 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482cad2-f06b-4a3d-b472-e844f49a42b6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:07.824839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:07.826968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:07.827352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:07.828054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:07.844009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:07.846066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:07.861672 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1344/2658] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:10:07.958942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c36d7-cfc5-4f62-b60d-5ccdb4ad55bd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:07.978750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c36d7-cfc5-4f62-b60d-5ccdb4ad55bd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:07.983356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:07.984662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:07.984954 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:07.985292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:08.052613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:08.054435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:08.065474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1315/2659] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:09.950628 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:09.955554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:09.994451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:10.143206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc518e69-c66a-4899-a768-cdc2a1a50496 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:10.160584 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:10.161245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:10.161525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:10.161787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] PATH_INFO: `/v3/users/eaef8f72768e4d1daf21257fdaf580d2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:10.162953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaef8f72768e4d1daf21257fdaf580d2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:10.165084 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:10.307918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaef8f72768e4d1daf21257fdaf580d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.321586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaef8f72768e4d1daf21257fdaf580d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.347650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaef8f72768e4d1daf21257fdaf580d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.348697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaef8f72768e4d1daf21257fdaf580d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.352795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fa867235-4ee5-43c5-9872-0dc26e6c3b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaef8f72768e4d1daf21257fdaf580d2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.379658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1345/2660] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:09 2022] DELETE /identity/v3/users/eaef8f72768e4d1daf21257fdaf580d2 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:10.648943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:10.671923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed28e-de3a-479f-92a0-9d3f546e7343 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:10.746107 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:10.760531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:10.761561 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:10.762001 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:10.762444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] PATH_INFO: `/v3/projects/8084c097adde47ffadcacca8f4d3ea73` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:10.763892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8084c097adde47ffadcacca8f4d3ea73)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:10.766571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:10.829489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8084c097adde47ffadcacca8f4d3ea73', 'request_id': 'req-ef3bec20-90b7-4469-894e-e712b80432f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.836270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8084c097adde47ffadcacca8f4d3ea73', 'request_id': 'req-ef3bec20-90b7-4469-894e-e712b80432f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.853502 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8084c097adde47ffadcacca8f4d3ea73', 'request_id': 'req-ef3bec20-90b7-4469-894e-e712b80432f5'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.870353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] The token cache is being invalidate because project 8084c097adde47ffadcacca8f4d3ea73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:10.870658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.877171 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ef3bec20-90b7-4469-894e-e712b80432f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:10.879633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1316/2661] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:10 2022] DELETE /identity/v3/projects/8084c097adde47ffadcacca8f4d3ea73 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:10.884953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301069eb-847f-47cd-9fe4-16035d5681f1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:10.885626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301069eb-847f-47cd-9fe4-16035d5681f1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:10.886210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301069eb-847f-47cd-9fe4-16035d5681f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:10.909994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-301069eb-847f-47cd-9fe4-16035d5681f1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:10.911725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-301069eb-847f-47cd-9fe4-16035d5681f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:11.003274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1346/2662] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:11.003958 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13400bb1-ff6f-4520-91b8-e5aac9c9dd2b None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.004249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13400bb1-ff6f-4520-91b8-e5aac9c9dd2b None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.004520 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13400bb1-ff6f-4520-91b8-e5aac9c9dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.037791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-13400bb1-ff6f-4520-91b8-e5aac9c9dd2b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:11.039376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-13400bb1-ff6f-4520-91b8-e5aac9c9dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:11.049244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1317/2663] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:11.054958 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301069eb-847f-47cd-9fe4-16035d5681f1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:11.067064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301069eb-847f-47cd-9fe4-16035d5681f1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:11.071728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:11.072335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.072610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.072880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.073791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:11.074101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:11.076773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:11.095333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1347/2664] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:10:11.096296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13400bb1-ff6f-4520-91b8-e5aac9c9dd2b None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:11.106740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13400bb1-ff6f-4520-91b8-e5aac9c9dd2b None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:11.112138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:11.112920 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.113204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.113480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.114117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:11.116040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:11.163576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1318/2665] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:11 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:10:11.164472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:11.177745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25340d4e-9005-4de7-b15e-8f3e38f4b6f0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:11.219635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:11.284277 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:11.285073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.285522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.285947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.286736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:11.289594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:11.361387 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1348/2666] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:11 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:11.363377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:11.392445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e4bb1-7387-4557-9cd7-8bafa9240464 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:11.409628 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:11.410299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.410583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.410866 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.411723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:11.413318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:11.443688 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1319/2667] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:11.445170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:11.460118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0755f-e087-4e86-9309-36f1cc7f7292 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:11.484060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9c1d0-9e61-448b-8b1f-4092edb55cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:11.485340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c1d0-9e61-448b-8b1f-4092edb55cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.485635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c1d0-9e61-448b-8b1f-4092edb55cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.487136 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c1d0-9e61-448b-8b1f-4092edb55cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.488753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9c1d0-9e61-448b-8b1f-4092edb55cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:11.503758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9c1d0-9e61-448b-8b1f-4092edb55cf0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:11.528098 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:11.534774 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1349/2668] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:11.543281 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92665bb0-227e-41d1-8916-e37560b764c0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:11.548732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:11.550645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.551010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.551367 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] PATH_INFO: `/v3/projects/f319bdd0037244749fb635d82729b782/users/26bf037a0dd84ca9b46d8e2693de8bc4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.573006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f319bdd0037244749fb635d82729b782, user_id=26bf037a0dd84ca9b46d8e2693de8bc4, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:11.576832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:11.602125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1320/2669] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:11 2022] PUT /identity/v3/projects/f319bdd0037244749fb635d82729b782/users/26bf037a0dd84ca9b46d8e2693de8bc4/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:11.603747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ef790-6a6c-49bc-a0a4-bb70fb088f8b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.604038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ef790-6a6c-49bc-a0a4-bb70fb088f8b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.604318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ef790-6a6c-49bc-a0a4-bb70fb088f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.632690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fc8ef790-6a6c-49bc-a0a4-bb70fb088f8b None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:11.636346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fc8ef790-6a6c-49bc-a0a4-bb70fb088f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:11.716693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1350/2670] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:11.731599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:11.731599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:11.795202 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c11c7c-751c-4100-8ce2-be1c75870117 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:11.799796 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873079b-959e-4bff-88d4-bee36d0320ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:11.800512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873079b-959e-4bff-88d4-bee36d0320ba service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:11.801104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873079b-959e-4bff-88d4-bee36d0320ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:11.801573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873079b-959e-4bff-88d4-bee36d0320ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:11.812436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873079b-959e-4bff-88d4-bee36d0320ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:11.814086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873079b-959e-4bff-88d4-bee36d0320ba service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:11.833456 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1321/2671] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:12.466346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:12.467405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:12.469687 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:12.494585 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] MFA Rules not processed for user `3cbefab6dddb45c381158126d203f66b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:12.496312 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:12.581091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1351/2672] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:13.395776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be766be-85f6-42e7-a67e-cefb6fdc2832 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:13.397180 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be766be-85f6-42e7-a67e-cefb6fdc2832 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:13.399177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be766be-85f6-42e7-a67e-cefb6fdc2832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:13.476336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-2be766be-85f6-42e7-a67e-cefb6fdc2832 None None] MFA Rules not processed for user `0752c7b0aa574a6ab87af239daeb178a`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:13.477922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2be766be-85f6-42e7-a67e-cefb6fdc2832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:13.500160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:13.591662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1322/2673] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:13.606764 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:13.608438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:13.683389 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:13.739436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06b239-f589-40a7-acfb-d9e3dc1a25bb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:13.751663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a8824-df0c-4e16-b010-668c015bf932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:13.752067 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a8824-df0c-4e16-b010-668c015bf932 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:13.752574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a8824-df0c-4e16-b010-668c015bf932 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:13.753639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a8824-df0c-4e16-b010-668c015bf932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:13.766041 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b65a8824-df0c-4e16-b010-668c015bf932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:13.859997 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a8824-df0c-4e16-b010-668c015bf932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:13.862626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a8824-df0c-4e16-b010-668c015bf932 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:13.879777 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1352/2674] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:15.677660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8d9c0-c4dc-4f1e-a5a5-9a8066db4685 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:15.678531 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8d9c0-c4dc-4f1e-a5a5-9a8066db4685 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:15.678987 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8d9c0-c4dc-4f1e-a5a5-9a8066db4685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:15.740499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7fc8d9c0-c4dc-4f1e-a5a5-9a8066db4685 None None] MFA Rules not processed for user `b339e5581cf044d9a55eb85561a58106`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:15.741767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7fc8d9c0-c4dc-4f1e-a5a5-9a8066db4685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:15.786825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1323/2675] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:18.930776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:18.931297 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:18.931522 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:18.969346 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] MFA Rules not processed for user `26bf037a0dd84ca9b46d8e2693de8bc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:18.970588 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:19.080377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1353/2676] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:19.088377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8d9c0-c4dc-4f1e-a5a5-9a8066db4685 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:19.122918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8d9c0-c4dc-4f1e-a5a5-9a8066db4685 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:19.134465 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea28787-bccf-4691-9f63-0fd6f41e0982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:19.135090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea28787-bccf-4691-9f63-0fd6f41e0982 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:19.135364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea28787-bccf-4691-9f63-0fd6f41e0982 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:19.135670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea28787-bccf-4691-9f63-0fd6f41e0982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:19.152538 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea28787-bccf-4691-9f63-0fd6f41e0982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:19.166588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea28787-bccf-4691-9f63-0fd6f41e0982 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:19.181186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1324/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:20.313832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:20.315279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:20.352479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4615-6358-4b52-84fc-b275fb0848f8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:20.359111 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:20.360117 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:20.360552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:20.360977 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] PATH_INFO: `/v3/users/5b887d6c3cf64e80a161eea6ec360ba9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:20.363273 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b887d6c3cf64e80a161eea6ec360ba9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:20.365256 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:20.458340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b887d6c3cf64e80a161eea6ec360ba9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:20.469245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b887d6c3cf64e80a161eea6ec360ba9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:20.483306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:20.494406 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b887d6c3cf64e80a161eea6ec360ba9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:20.495793 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b887d6c3cf64e80a161eea6ec360ba9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:20.496691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b887d6c3cf64e80a161eea6ec360ba9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:20.523179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1354/2678] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:20 2022] DELETE /identity/v3/users/5b887d6c3cf64e80a161eea6ec360ba9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:10:20.529480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6296cb-ccb9-4ea1-8fea-5f65841beff8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:20.530036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6296cb-ccb9-4ea1-8fea-5f65841beff8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:20.530769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6296cb-ccb9-4ea1-8fea-5f65841beff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:20.556054 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5e6296cb-ccb9-4ea1-8fea-5f65841beff8 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:20.557491 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5e6296cb-ccb9-4ea1-8fea-5f65841beff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:20.634121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1325/2679] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:20.648384 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:20.650089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:20.751371 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf14a5-71c6-4e6e-bed3-d447e2338e8b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:20.757244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:20.757867 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:20.758152 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:20.758421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:20.768633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:20.770113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:20.784468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1355/2680] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:10:21.130378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b9e0f-d652-457a-af3e-7d8dee05a10a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:21.131051 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b9e0f-d652-457a-af3e-7d8dee05a10a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:21.132812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b9e0f-d652-457a-af3e-7d8dee05a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:21.173325 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-129b9e0f-d652-457a-af3e-7d8dee05a10a None None] MFA Rules not processed for user `26bf037a0dd84ca9b46d8e2693de8bc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:21.175176 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-129b9e0f-d652-457a-af3e-7d8dee05a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:21.253252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1326/2681] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:21.325025 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:21.327731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:21.367589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ac819-2b68-42dd-90ed-409a2e578b51 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:21.390202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:21.391238 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:21.391919 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:21.393813 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] PATH_INFO: `/v3/projects/7cf763f59a3447a497ee0e22e3a1bcca` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:21.398442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf763f59a3447a497ee0e22e3a1bcca)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:21.406515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:21.551571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf763f59a3447a497ee0e22e3a1bcca', 'request_id': 'req-bd96f919-263f-4f73-b858-5789954b98c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.562597 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf763f59a3447a497ee0e22e3a1bcca', 'request_id': 'req-bd96f919-263f-4f73-b858-5789954b98c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.576048 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf763f59a3447a497ee0e22e3a1bcca', 'request_id': 'req-bd96f919-263f-4f73-b858-5789954b98c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.598980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] The token cache is being invalidate because project 7cf763f59a3447a497ee0e22e3a1bcca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:21.602349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.603996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.616677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1356/2682] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:21 2022] DELETE /identity/v3/projects/7cf763f59a3447a497ee0e22e3a1bcca => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:21.618585 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b9e0f-d652-457a-af3e-7d8dee05a10a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:21.641295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b9e0f-d652-457a-af3e-7d8dee05a10a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:21.745715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=100 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:21.802661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:21.803326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:21.803657 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:21.803933 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] PATH_INFO: `/v3/users/c338286b8ffc414694cd3a21348cf913` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:21.805808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c338286b8ffc414694cd3a21348cf913)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:21.808182 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:21.905348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c338286b8ffc414694cd3a21348cf913'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.906326 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c338286b8ffc414694cd3a21348cf913'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.926244 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c338286b8ffc414694cd3a21348cf913'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.948879 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c338286b8ffc414694cd3a21348cf913'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.952010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-122d82ad-0339-4b85-b72d-14f02f812760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c338286b8ffc414694cd3a21348cf913'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:21.984374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1327/2683] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:21 2022] DELETE /identity/v3/users/c338286b8ffc414694cd3a21348cf913 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:22.613354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:22.625817 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:22.652858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96f919-263f-4f73-b858-5789954b98c2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:22.744009 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:22.744890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:22.745177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:22.745440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] PATH_INFO: `/v3/projects/4224481659d94636b257d09b1860b2c9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:22.746595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4224481659d94636b257d09b1860b2c9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:22.748573 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:22.762973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:22.817884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4224481659d94636b257d09b1860b2c9', 'request_id': 'req-67cb106e-3a96-4de0-bc6a-8ddc726e870a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:22.826066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4224481659d94636b257d09b1860b2c9', 'request_id': 'req-67cb106e-3a96-4de0-bc6a-8ddc726e870a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:22.832827 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4224481659d94636b257d09b1860b2c9', 'request_id': 'req-67cb106e-3a96-4de0-bc6a-8ddc726e870a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:22.847635 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] The token cache is being invalidate because project 4224481659d94636b257d09b1860b2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:22.848173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:22.848898 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-67cb106e-3a96-4de0-bc6a-8ddc726e870a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:22.850790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1357/2684] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:22 2022] DELETE /identity/v3/projects/4224481659d94636b257d09b1860b2c9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:22.856090 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301c6b-e3ee-4b96-a897-2d61123c4fbc None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:22.856947 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301c6b-e3ee-4b96-a897-2d61123c4fbc None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:22.857380 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301c6b-e3ee-4b96-a897-2d61123c4fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:22.888103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f5301c6b-e3ee-4b96-a897-2d61123c4fbc None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:22.889405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f5301c6b-e3ee-4b96-a897-2d61123c4fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:22.959847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1328/2685] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:22.968367 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c26c48-7583-4f92-947d-15d1c90de85c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:22.968663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c26c48-7583-4f92-947d-15d1c90de85c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:22.969425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c26c48-7583-4f92-947d-15d1c90de85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:22.991789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-53c26c48-7583-4f92-947d-15d1c90de85c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:22.993788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-53c26c48-7583-4f92-947d-15d1c90de85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:23.002349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1358/2686] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:23.007607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301c6b-e3ee-4b96-a897-2d61123c4fbc None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:23.018084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301c6b-e3ee-4b96-a897-2d61123c4fbc None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:23.023048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:23.023795 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.024076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.024455 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.025102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:23.025405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:23.026993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:23.039775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1329/2687] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:10:23.044392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c26c48-7583-4f92-947d-15d1c90de85c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:23.054967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c26c48-7583-4f92-947d-15d1c90de85c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:23.059744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:23.060358 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.060675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.060971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.061647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:23.063644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:23.099537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1359/2688] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:23 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:23.101922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:23.165680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be25da5-5c02-4d96-9c1a-4c85813aad25 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:23.232929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:23.265929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:23.266646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.267002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.267371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.268543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:23.272138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:23.310401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1330/2689] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:23 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:23.311929 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:23.328702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43e49b-87d3-4fef-9087-3dd08df9384e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:23.335735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:23.336504 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.336790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.337116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.337845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:23.339300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:23.354018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1360/2690] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:23.355317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:23.365646 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779167e-1cea-4926-98c2-f532004b0fda admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:23.370360 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438100c6-d272-4c77-b229-c1d36eb760fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:23.370979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438100c6-d272-4c77-b229-c1d36eb760fc admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.371259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438100c6-d272-4c77-b229-c1d36eb760fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.371559 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438100c6-d272-4c77-b229-c1d36eb760fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.372204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438100c6-d272-4c77-b229-c1d36eb760fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:23.374459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438100c6-d272-4c77-b229-c1d36eb760fc admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:23.390748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1331/2691] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:23.392563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:23.407468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc010d8c-690c-4d0b-8d65-76ddbfa18113 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:23.414544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:23.415353 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.415790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.416014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] PATH_INFO: `/v3/projects/22e891e32ba74bd6bc86e3dedcf31280/users/5ea3bf5707574f5b8ab84b87e3391c1b/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.443326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e891e32ba74bd6bc86e3dedcf31280, user_id=5ea3bf5707574f5b8ab84b87e3391c1b, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:23.446532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:23.468201 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1361/2692] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:23 2022] PUT /identity/v3/projects/22e891e32ba74bd6bc86e3dedcf31280/users/5ea3bf5707574f5b8ab84b87e3391c1b/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:23.476316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da05d1-4ade-4328-9376-a7f102413ab6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.476742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da05d1-4ade-4328-9376-a7f102413ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.477120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da05d1-4ade-4328-9376-a7f102413ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.534038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-97da05d1-4ade-4328-9376-a7f102413ab6 None None] MFA Rules not processed for user `5ea3bf5707574f5b8ab84b87e3391c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:23.535327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-97da05d1-4ade-4328-9376-a7f102413ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:23.685399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1332/2693] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:23.690521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:23.840372 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b127ff-00a9-4d5e-b1fb-00dacb93e30a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:23.848524 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:23.849251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:23.849605 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:23.850060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:23.850936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:23.854674 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:23.970818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1362/2694] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:23 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:23.974930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da05d1-4ade-4328-9376-a7f102413ab6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.005668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da05d1-4ade-4328-9376-a7f102413ab6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.103371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.105015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.105500 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.107371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.109894 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.115663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.154539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1333/2695] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:23 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:10:24.158753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.177991 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f631ca-800d-4714-97dd-e723293c6e50 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.189054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.189814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.190101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.190375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.191028 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.192895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.205961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1363/2696] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:24.212341 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.246859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6752e69-5166-4e77-9d9e-3d59af55796d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.253038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a156dd-ecf7-43d5-9ca6-c310777dad17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.253656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a156dd-ecf7-43d5-9ca6-c310777dad17 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.253935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a156dd-ecf7-43d5-9ca6-c310777dad17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.254206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a156dd-ecf7-43d5-9ca6-c310777dad17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.254851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a156dd-ecf7-43d5-9ca6-c310777dad17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.256674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a156dd-ecf7-43d5-9ca6-c310777dad17 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.267183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1334/2697] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:24.272662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.283138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8badd06-01a7-4c42-b1a4-c185961e29d0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.288271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.288973 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.289622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.289622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] PATH_INFO: `/v3/projects/a9427b2ae1e14e56bf7ae31ff12cef54/users/f2de8f632d07454f914ba57ac0cad948/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.299132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:24.307138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9427b2ae1e14e56bf7ae31ff12cef54, user_id=f2de8f632d07454f914ba57ac0cad948, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.309826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.339528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1364/2698] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:24 2022] PUT /identity/v3/projects/a9427b2ae1e14e56bf7ae31ff12cef54/users/f2de8f632d07454f914ba57ac0cad948/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:24.343524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b4ff-33e8-4879-8d89-06be25e3c50c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.343896 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b4ff-33e8-4879-8d89-06be25e3c50c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.344243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b4ff-33e8-4879-8d89-06be25e3c50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.375669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-aa76b4ff-33e8-4879-8d89-06be25e3c50c None None] MFA Rules not processed for user `f2de8f632d07454f914ba57ac0cad948`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:24.377301 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aa76b4ff-33e8-4879-8d89-06be25e3c50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:24.393692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:24.468993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1335/2699] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:24.473485 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.541951 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db66b-e796-4bdd-ad7d-84f47a59d062 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.547014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.547652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.547938 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.548212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.548895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.550788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.582746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1365/2700] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:24 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:24.587029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76b4ff-33e8-4879-8d89-06be25e3c50c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.605387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76b4ff-33e8-4879-8d89-06be25e3c50c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.702028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.703085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.703085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.703294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.703880 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.706002 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.742153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1336/2701] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:24 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:24.742682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.754600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac801-2701-4903-9bdf-68a0ce9aee17 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.767811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.768839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.769327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.769789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.770643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.772464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.806476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1366/2702] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:24.807435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.824205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72079a2-df45-4a19-a462-073a2e56609d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.829314 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.829992 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.830445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.830728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] PATH_INFO: `/v3/projects/bf696793dc8c490b9b53d78f102ffbb1/users/7fdd2a15003845219a8501152891dffd/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.847511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf696793dc8c490b9b53d78f102ffbb1, user_id=7fdd2a15003845219a8501152891dffd, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.850187 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.867825 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1337/2703] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:24 2022] PUT /identity/v3/projects/bf696793dc8c490b9b53d78f102ffbb1/users/7fdd2a15003845219a8501152891dffd/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:24.871877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.885836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2561b6-53ea-49dc-b9f5-7bfdee928e00 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.952145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b378003-5c02-474b-8dd2-628ea05e63ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.952763 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b378003-5c02-474b-8dd2-628ea05e63ca admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.953042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b378003-5c02-474b-8dd2-628ea05e63ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.953474 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b378003-5c02-474b-8dd2-628ea05e63ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.954129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b378003-5c02-474b-8dd2-628ea05e63ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:24.956112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b378003-5c02-474b-8dd2-628ea05e63ca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:24.971308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1367/2704] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:10:24.975578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:24.989718 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc94e7-25ac-461d-8a06-eace5fb9acc6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:24.994451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:24.995106 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:24.995658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:24.995805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] PATH_INFO: `/v3/projects/bf696793dc8c490b9b53d78f102ffbb1/users/7fdd2a15003845219a8501152891dffd/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:24.997813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf696793dc8c490b9b53d78f102ffbb1, user_id=7fdd2a15003845219a8501152891dffd, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:25.001368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:25.013056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1338/2705] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:24 2022] PUT /identity/v3/projects/bf696793dc8c490b9b53d78f102ffbb1/users/7fdd2a15003845219a8501152891dffd/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:25.023014 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:25.023385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:25.023789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:25.054397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] MFA Rules not processed for user `7fdd2a15003845219a8501152891dffd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:25.055850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:25.109633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:25.139537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1368/2706] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:25.149620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:25.151615 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:25.234462 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72688f-d4b4-48c6-81a5-ad88ba501079 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:25.244472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:25.246444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:25.247257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:25.248419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:25.262288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:25.297486 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:25.299378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:25.313379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1339/2707] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:25.932112 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:25.933600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:25.975343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f0798-8b63-429f-870f-52fadceca510 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:25.979042 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:25.979846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:25.980126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:25.980396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:26.045859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:26.047296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:26.056885 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1369/2708] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:30.143935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:30.159474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb9ac-8b93-4dd7-8397-bd4891d32ad5 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:30.163109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246a9ab-5aa8-4c52-8428-889b96571a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:30.163735 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246a9ab-5aa8-4c52-8428-889b96571a6e service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:30.164016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246a9ab-5aa8-4c52-8428-889b96571a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:30.164289 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246a9ab-5aa8-4c52-8428-889b96571a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:30.225881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246a9ab-5aa8-4c52-8428-889b96571a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:30.227324 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246a9ab-5aa8-4c52-8428-889b96571a6e service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:30.237241 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1340/2709] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:32.475789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:32.563626 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85930255-ecf4-4231-9c2c-79963b40750c service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:32.568249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:32.568877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:32.569153 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:32.569423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] PATH_INFO: `/v3/users/5ea3bf5707574f5b8ab84b87e3391c1b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:32.570536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea3bf5707574f5b8ab84b87e3391c1b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:32.572488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:32.611692 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:32.620904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea3bf5707574f5b8ab84b87e3391c1b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:32.628541 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea3bf5707574f5b8ab84b87e3391c1b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:32.640482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea3bf5707574f5b8ab84b87e3391c1b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:32.641210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea3bf5707574f5b8ab84b87e3391c1b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:32.641816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea3bf5707574f5b8ab84b87e3391c1b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:32.656548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1370/2710] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:32 2022] DELETE /identity/v3/users/5ea3bf5707574f5b8ab84b87e3391c1b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:32.661010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:32.661484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:32.661613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:32.702728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:32.704083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:32.771464 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1341/2711] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:32.781943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:32.783443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:32.850493 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd7708-42c6-4bad-a2c7-521407224384 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:32.854347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:32.855029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:32.855399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:32.855798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:32.866024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:32.867515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:32.879228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1371/2712] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:33.231101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:33.232600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:33.250917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b351-9a20-41c2-848b-9fe97bb875bd None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:33.256448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:33.258064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:33.258448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:33.258823 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] PATH_INFO: `/v3/projects/22e891e32ba74bd6bc86e3dedcf31280` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:33.260122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e891e32ba74bd6bc86e3dedcf31280)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:33.262817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:33.330877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:33.332530 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e891e32ba74bd6bc86e3dedcf31280', 'request_id': 'req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.340438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e891e32ba74bd6bc86e3dedcf31280', 'request_id': 'req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.354586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e891e32ba74bd6bc86e3dedcf31280', 'request_id': 'req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.366786 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] The token cache is being invalidate because project 22e891e32ba74bd6bc86e3dedcf31280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:33.367088 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.367791 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.369767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1342/2713] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:33 2022] DELETE /identity/v3/projects/22e891e32ba74bd6bc86e3dedcf31280 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:33.373758 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:33.388743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73173d16-ef06-4598-8fce-62ee23153922 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:33.463961 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:33.464804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:33.465188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:33.465563 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] PATH_INFO: `/v3/users/f2de8f632d07454f914ba57ac0cad948` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:33.466933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2de8f632d07454f914ba57ac0cad948)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:33.469145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:33.530941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2de8f632d07454f914ba57ac0cad948'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.543210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2de8f632d07454f914ba57ac0cad948'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.545248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:33.559385 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2de8f632d07454f914ba57ac0cad948'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.560407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2de8f632d07454f914ba57ac0cad948'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.561574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2de8f632d07454f914ba57ac0cad948'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:33.579055 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1372/2714] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:33 2022] DELETE /identity/v3/users/f2de8f632d07454f914ba57ac0cad948 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:34.214121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:34.215676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:34.244882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57a6c3-f2b8-4585-88ab-bd03f4c2d866 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:34.318710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:34.319586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:34.319864 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:34.320262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] PATH_INFO: `/v3/projects/a9427b2ae1e14e56bf7ae31ff12cef54` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:34.321422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9427b2ae1e14e56bf7ae31ff12cef54)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:34.323588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:34.376807 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9427b2ae1e14e56bf7ae31ff12cef54', 'request_id': 'req-facfb9e0-3df3-4897-857e-bb2245a11906'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.384355 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9427b2ae1e14e56bf7ae31ff12cef54', 'request_id': 'req-facfb9e0-3df3-4897-857e-bb2245a11906'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.398094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9427b2ae1e14e56bf7ae31ff12cef54', 'request_id': 'req-facfb9e0-3df3-4897-857e-bb2245a11906'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.414253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] The token cache is being invalidate because project a9427b2ae1e14e56bf7ae31ff12cef54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:34.414861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.415647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.418004 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1343/2715] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:34 2022] DELETE /identity/v3/projects/a9427b2ae1e14e56bf7ae31ff12cef54 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:34.422532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:34.424441 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:34.438789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401060c-e023-4beb-a4ed-02773c904a60 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:34.509549 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:34.510247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:34.510526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:34.510971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] PATH_INFO: `/v3/users/7fdd2a15003845219a8501152891dffd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:34.512842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fdd2a15003845219a8501152891dffd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:34.515039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:34.572670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fdd2a15003845219a8501152891dffd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.582419 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fdd2a15003845219a8501152891dffd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.617361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fdd2a15003845219a8501152891dffd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.618217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fdd2a15003845219a8501152891dffd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.618949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b1cfa335-0076-459a-83ca-43dc473c0fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fdd2a15003845219a8501152891dffd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:34.646828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1373/2716] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:34 2022] DELETE /identity/v3/users/7fdd2a15003845219a8501152891dffd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:35.137684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.139648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:35.164068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfb9e0-3df3-4897-857e-bb2245a11906 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:35.281524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:35.283184 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.283643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.283997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] PATH_INFO: `/v3/projects/bf696793dc8c490b9b53d78f102ffbb1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.286229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf696793dc8c490b9b53d78f102ffbb1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:35.290449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:35.305354 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:35.369146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf696793dc8c490b9b53d78f102ffbb1', 'request_id': 'req-858bf103-7303-4f39-a0c3-cafe689f290f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:35.384836 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf696793dc8c490b9b53d78f102ffbb1', 'request_id': 'req-858bf103-7303-4f39-a0c3-cafe689f290f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:35.402976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf696793dc8c490b9b53d78f102ffbb1', 'request_id': 'req-858bf103-7303-4f39-a0c3-cafe689f290f'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:35.421170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] The token cache is being invalidate because project bf696793dc8c490b9b53d78f102ffbb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:35.421608 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:35.422333 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-858bf103-7303-4f39-a0c3-cafe689f290f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:35.427532 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1344/2717] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:35 2022] DELETE /identity/v3/projects/bf696793dc8c490b9b53d78f102ffbb1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:35.430359 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4043a-53c5-49c0-a96f-3522f78748e1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.430728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4043a-53c5-49c0-a96f-3522f78748e1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.430916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4043a-53c5-49c0-a96f-3522f78748e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.489529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-27d4043a-53c5-49c0-a96f-3522f78748e1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:35.491360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-27d4043a-53c5-49c0-a96f-3522f78748e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:35.533058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:35.585711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1374/2718] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:35.596075 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6937aa-4ff8-4d54-aeaa-c97399938aa9 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.596624 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6937aa-4ff8-4d54-aeaa-c97399938aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.597368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6937aa-4ff8-4d54-aeaa-c97399938aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.624308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3e6937aa-4ff8-4d54-aeaa-c97399938aa9 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:35.625893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3e6937aa-4ff8-4d54-aeaa-c97399938aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:35.641403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1345/2719] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:35.642845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4043a-53c5-49c0-a96f-3522f78748e1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.653840 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4043a-53c5-49c0-a96f-3522f78748e1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:35.659816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:35.660515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.660870 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.661210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.662109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:35.662481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:35.664218 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:35.679146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1375/2720] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:10:35.683365 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6937aa-4ff8-4d54-aeaa-c97399938aa9 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.692897 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6937aa-4ff8-4d54-aeaa-c97399938aa9 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:35.697888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:35.698496 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.698774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.699044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.699726 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:35.701672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:35.746018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1346/2721] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:35 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:35.746688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.757788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa543-e5af-4952-a005-fe1341daecea admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:35.820013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:35.820664 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.820947 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.821215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.822133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:35.824074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:35.848805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1376/2722] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:35 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:35.852762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.864596 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c42be-006c-42e4-aaaf-bde84630be00 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:35.870728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:35.871419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.872122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.872861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.873134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:35.874828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:35.886689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1347/2723] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:10:35.891536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.904081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1b64e-1012-4c25-99fd-90d660776f36 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:35.912273 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.912486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:35.913740 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.914124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.914502 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.915329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:35.915989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:35.917866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:35.940005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1377/2724] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:35.941462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:35.952937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e9c9-3c7b-49e1-8d9f-0649a7f93cbb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:35.958754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb2212-2a8c-4f3e-a497-ffcd555cf815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:35.959715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb2212-2a8c-4f3e-a497-ffcd555cf815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:35.960202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb2212-2a8c-4f3e-a497-ffcd555cf815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:35.960691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb2212-2a8c-4f3e-a497-ffcd555cf815 admin admin] PATH_INFO: `/v3/projects/7d030926c1bf4e7ea071fceefb9842de/users/a3da07149aad46ec94f31a79c6e556ae/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:35.984130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb2212-2a8c-4f3e-a497-ffcd555cf815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d030926c1bf4e7ea071fceefb9842de, user_id=a3da07149aad46ec94f31a79c6e556ae, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:35.987181 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb2212-2a8c-4f3e-a497-ffcd555cf815 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.006321 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8801d4-d4b5-409f-bc22-a525b20e1fba admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.010276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1378/2725] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:35 2022] PUT /identity/v3/projects/7d030926c1bf4e7ea071fceefb9842de/users/a3da07149aad46ec94f31a79c6e556ae/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:36.022458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f820-a704-43bb-a9f6-f8acdf77fedc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.022866 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f820-a704-43bb-a9f6-f8acdf77fedc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.023159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f820-a704-43bb-a9f6-f8acdf77fedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.057242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6eb0f820-a704-43bb-a9f6-f8acdf77fedc None None] MFA Rules not processed for user `a3da07149aad46ec94f31a79c6e556ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:36.058462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6eb0f820-a704-43bb-a9f6-f8acdf77fedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:36.073535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.074190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.074426 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.074709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.077344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:36.115423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:36.132281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1379/2726] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:36.133230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f820-a704-43bb-a9f6-f8acdf77fedc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:36.137845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:36.191044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:36.192524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.212300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1348/2727] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:10:36.239660 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f820-a704-43bb-a9f6-f8acdf77fedc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.248029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.249129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.249826 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.250281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.251230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:36.254314 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.315580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1380/2728] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:36 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:36.320478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:36.335594 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3901ea-8f26-4f28-b401-8fdd7c7dd1c0 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.447640 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.448387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.448769 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.449147 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.449960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:36.453077 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.487588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1349/2729] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:36 2022] POST /identity/v3/users => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:36.491854 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:36.504442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a173-13af-4f46-b6a7-62a9b33ea7a9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.509495 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.510107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.510383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.510653 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.511331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:36.513079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.526974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1381/2730] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:36.531847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:36.542007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ae617-06cc-447e-a519-9e219ab405d0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.553330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.553781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.554665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.554665 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] PATH_INFO: `/v3/projects/739d27a514c54c078f83e0fd3c60d509/users/223e0d5459634a4fafccae5a480d91dc/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.578215 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=739d27a514c54c078f83e0fd3c60d509, user_id=223e0d5459634a4fafccae5a480d91dc, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:36.581160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.599738 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1350/2731] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:36 2022] PUT /identity/v3/projects/739d27a514c54c078f83e0fd3c60d509/users/223e0d5459634a4fafccae5a480d91dc/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:36.603971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:36.618255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c9e76-c407-4ee0-9f6b-0215176b9693 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.694501 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ed9d4-a7d0-47b8-8ae4-e6264924b079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.695115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ed9d4-a7d0-47b8-8ae4-e6264924b079 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.695390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ed9d4-a7d0-47b8-8ae4-e6264924b079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.695690 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ed9d4-a7d0-47b8-8ae4-e6264924b079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.696332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ed9d4-a7d0-47b8-8ae4-e6264924b079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:36.697920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ed9d4-a7d0-47b8-8ae4-e6264924b079 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.713846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1382/2732] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:36.716136 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:36.731868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6033-3931-4ac0-a953-5405330fae58 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.736000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.736678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.737721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.737996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] PATH_INFO: `/v3/projects/739d27a514c54c078f83e0fd3c60d509/users/223e0d5459634a4fafccae5a480d91dc/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.740030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=739d27a514c54c078f83e0fd3c60d509, user_id=223e0d5459634a4fafccae5a480d91dc, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:36.743667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:36.758654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1351/2733] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:36 2022] PUT /identity/v3/projects/739d27a514c54c078f83e0fd3c60d509/users/223e0d5459634a4fafccae5a480d91dc/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:36.769544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270588-3854-4dd0-a6bc-4cf1961d2c23 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.769904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270588-3854-4dd0-a6bc-4cf1961d2c23 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.770454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270588-3854-4dd0-a6bc-4cf1961d2c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.806292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-43270588-3854-4dd0-a6bc-4cf1961d2c23 None None] MFA Rules not processed for user `223e0d5459634a4fafccae5a480d91dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:36.807537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43270588-3854-4dd0-a6bc-4cf1961d2c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:36.883125 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:36.889002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1383/2734] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:36.901711 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:36.905775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:36.972980 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9503e-2026-416c-bc33-21d75315c707 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:36.978011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:36.979399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:36.979399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:36.979399 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:36.999712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:37.001826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:37.023409 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1352/2735] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:37.183420 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43270588-3854-4dd0-a6bc-4cf1961d2c23 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:37.196652 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43270588-3854-4dd0-a6bc-4cf1961d2c23 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:37.200685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdbdfc-dfb0-41aa-bc7f-edca85bda8a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:37.201398 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbdfc-dfb0-41aa-bc7f-edca85bda8a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:37.201725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbdfc-dfb0-41aa-bc7f-edca85bda8a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:37.202076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbdfc-dfb0-41aa-bc7f-edca85bda8a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:37.306089 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdbdfc-dfb0-41aa-bc7f-edca85bda8a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:37.307677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdbdfc-dfb0-41aa-bc7f-edca85bda8a6 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:37.318871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1384/2736] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:40.090780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:40.109893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:40.163955 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264ee36-f1bb-4c21-91d7-ec57bf460229 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:40.168521 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:40.169181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:40.169463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:40.169739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] PATH_INFO: `/v3/users/a3da07149aad46ec94f31a79c6e556ae` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:40.171438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3da07149aad46ec94f31a79c6e556ae)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:40.173357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:40.220680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3da07149aad46ec94f31a79c6e556ae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.221243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3da07149aad46ec94f31a79c6e556ae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.228637 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3da07149aad46ec94f31a79c6e556ae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.241076 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3da07149aad46ec94f31a79c6e556ae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.241793 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3da07149aad46ec94f31a79c6e556ae'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.256181 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1353/2737] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:40 2022] DELETE /identity/v3/users/a3da07149aad46ec94f31a79c6e556ae => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:10:40.260710 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:40.261276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:40.261276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:40.280488 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:40.281715 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:40.344242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1385/2738] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:40.354013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:40.355376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:40.413444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda896a-cedd-43ea-8c4d-81276376c88e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:40.417037 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:40.417631 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:40.417908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:40.418175 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:40.427650 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:40.429056 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:40.438519 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1354/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:10:40.709421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:40.710868 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:40.723944 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05b2f7-c847-4639-8a86-69c33d9616e4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:40.729060 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:40.729705 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:40.729988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:40.730264 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] PATH_INFO: `/v3/projects/7d030926c1bf4e7ea071fceefb9842de` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:40.731382 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d030926c1bf4e7ea071fceefb9842de)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:40.733850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:40.781249 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d030926c1bf4e7ea071fceefb9842de', 'request_id': 'req-c52cd348-4e99-4829-a6ce-ef61382c45dd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.787340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d030926c1bf4e7ea071fceefb9842de', 'request_id': 'req-c52cd348-4e99-4829-a6ce-ef61382c45dd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.793066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d030926c1bf4e7ea071fceefb9842de', 'request_id': 'req-c52cd348-4e99-4829-a6ce-ef61382c45dd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.806132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] The token cache is being invalidate because project 7d030926c1bf4e7ea071fceefb9842de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:40.806421 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.807026 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.808564 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1386/2740] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:40 2022] DELETE /identity/v3/projects/7d030926c1bf4e7ea071fceefb9842de => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:40.813332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:40.823007 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa70f2-05c5-4a2f-810f-bdf5854ef97a service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:40.859797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:40.880072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:40.880709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:40.880991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:40.881268 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] PATH_INFO: `/v3/users/223e0d5459634a4fafccae5a480d91dc` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:40.883102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=223e0d5459634a4fafccae5a480d91dc)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:40.886751 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:40.943681 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223e0d5459634a4fafccae5a480d91dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.944336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223e0d5459634a4fafccae5a480d91dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.951170 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223e0d5459634a4fafccae5a480d91dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.962822 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223e0d5459634a4fafccae5a480d91dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.963448 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-f4e8006c-250f-47e6-8065-518234870bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223e0d5459634a4fafccae5a480d91dc'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:40.977661 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1355/2741] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:40 2022] DELETE /identity/v3/users/223e0d5459634a4fafccae5a480d91dc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:41.247115 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.248791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:41.262145 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd348-4e99-4829-a6ce-ef61382c45dd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:41.280489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:41.321670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:41.322217 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.322496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.322765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] PATH_INFO: `/v3/projects/739d27a514c54c078f83e0fd3c60d509` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.323933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=739d27a514c54c078f83e0fd3c60d509)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:41.325877 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:41.376998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '739d27a514c54c078f83e0fd3c60d509', 'request_id': 'req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:41.383603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '739d27a514c54c078f83e0fd3c60d509', 'request_id': 'req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:41.391215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '739d27a514c54c078f83e0fd3c60d509', 'request_id': 'req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:41.408396 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] The token cache is being invalidate because project 739d27a514c54c078f83e0fd3c60d509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:41.408685 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:41.409334 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8e8b776-11fd-4d4d-ac6a-813551cd2abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:41.410792 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1387/2742] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:41 2022] DELETE /identity/v3/projects/739d27a514c54c078f83e0fd3c60d509 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:41.415584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcb17b-1485-4f21-a21f-d283f0c8f78f None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.416097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcb17b-1485-4f21-a21f-d283f0c8f78f None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.416199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcb17b-1485-4f21-a21f-d283f0c8f78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.436336 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-16fcb17b-1485-4f21-a21f-d283f0c8f78f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:41.437536 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-16fcb17b-1485-4f21-a21f-d283f0c8f78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:41.500797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1356/2743] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:41.508437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339621-7586-4788-af45-c89252eb70d1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.508725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339621-7586-4788-af45-c89252eb70d1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.508996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339621-7586-4788-af45-c89252eb70d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.532551 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c8339621-7586-4788-af45-c89252eb70d1 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:41.533873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c8339621-7586-4788-af45-c89252eb70d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:41.542397 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1388/2744] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:41.546619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcb17b-1485-4f21-a21f-d283f0c8f78f None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.557010 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcb17b-1485-4f21-a21f-d283f0c8f78f None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:41.561851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:41.562438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.562712 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.562979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.563645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:41.563942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:41.565509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:41.577862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1357/2745] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:10:41.582426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339621-7586-4788-af45-c89252eb70d1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.591896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339621-7586-4788-af45-c89252eb70d1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:41.597126 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:41.597775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.597996 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.598267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.598890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:41.600804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:41.631400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1389/2746] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:41 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:41.632601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.643100 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20b079-aa04-40e8-b20c-24a265e1fc09 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:41.689497 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:41.705029 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:41.705772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.706122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.706485 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.707188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:41.709236 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:41.736979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1358/2747] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:41 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:10:41.741135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.751913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e090146-2303-4bcd-9c33-1019bd952ec1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:41.756525 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:41.757120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.757400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.757669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.758332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:41.759950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:41.770887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1390/2748] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:41.774676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.785503 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514dfab-77d7-4c44-971f-7d61c566d676 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:41.790762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37612ed4-2c9c-4668-b570-3f691555a24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:41.791550 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37612ed4-2c9c-4668-b570-3f691555a24a admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.791716 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37612ed4-2c9c-4668-b570-3f691555a24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.791989 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37612ed4-2c9c-4668-b570-3f691555a24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.792633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37612ed4-2c9c-4668-b570-3f691555a24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:41.794221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37612ed4-2c9c-4668-b570-3f691555a24a admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:41.805418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1359/2749] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:10:41.809540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.819976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2da58a-a396-4794-a3e0-8fee770b52cf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:41.824880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:41.825519 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.825800 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.826071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] PATH_INFO: `/v3/projects/1423919b778544d990acb90eaf504279/users/3d8e1de14e5547ac83b3802c79650cf0/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.841465 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1423919b778544d990acb90eaf504279, user_id=3d8e1de14e5547ac83b3802c79650cf0, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:41.844116 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:41.856330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1391/2750] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:41 2022] PUT /identity/v3/projects/1423919b778544d990acb90eaf504279/users/3d8e1de14e5547ac83b3802c79650cf0/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:41.866459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e266b-85f0-491b-a440-7842fb7c4eb1 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:41.866743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e266b-85f0-491b-a440-7842fb7c4eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:41.867015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e266b-85f0-491b-a440-7842fb7c4eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:41.895744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5e9e266b-85f0-491b-a440-7842fb7c4eb1 None None] MFA Rules not processed for user `3d8e1de14e5547ac83b3802c79650cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:41.896937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5e9e266b-85f0-491b-a440-7842fb7c4eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:41.959649 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1360/2751] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:41.970663 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:41.973049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:42.025601 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:42.032694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972673-d066-4e6b-915a-da1c2ba56a8a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:42.036666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1e290-8338-461f-a802-95d9dcdb6334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:42.037253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1e290-8338-461f-a802-95d9dcdb6334 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:42.037532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1e290-8338-461f-a802-95d9dcdb6334 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:42.037797 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1e290-8338-461f-a802-95d9dcdb6334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:42.046337 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1e290-8338-461f-a802-95d9dcdb6334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:42.047795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1e290-8338-461f-a802-95d9dcdb6334 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:42.056562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1392/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:42.078588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e266b-85f0-491b-a440-7842fb7c4eb1 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:42.139401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e266b-85f0-491b-a440-7842fb7c4eb1 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:42.143851 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:42.144469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:42.144746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:42.145019 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] PATH_INFO: `/v3/users/3d8e1de14e5547ac83b3802c79650cf0` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:42.146142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8e1de14e5547ac83b3802c79650cf0)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:42.148097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:42.192475 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8e1de14e5547ac83b3802c79650cf0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.193126 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8e1de14e5547ac83b3802c79650cf0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.199895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8e1de14e5547ac83b3802c79650cf0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.200994 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:42.212971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8e1de14e5547ac83b3802c79650cf0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.213622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8e1de14e5547ac83b3802c79650cf0'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.228143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1361/2753] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:42 2022] DELETE /identity/v3/users/3d8e1de14e5547ac83b3802c79650cf0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:42.233195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:42.233486 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:42.233762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:42.253033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:42.254458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:42.312481 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1393/2754] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:42.322086 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:42.323535 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:42.381096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f8d2-aaf1-4d89-8731-28a5f3a16836 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:42.384723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5f64a-3945-4e16-b553-2a86f95e70e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:42.385330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5f64a-3945-4e16-b553-2a86f95e70e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:42.385638 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5f64a-3945-4e16-b553-2a86f95e70e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:42.385983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5f64a-3945-4e16-b553-2a86f95e70e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:42.397271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5f64a-3945-4e16-b553-2a86f95e70e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:42.398655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5f64a-3945-4e16-b553-2a86f95e70e4 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:42.408667 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1362/2755] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:42.767135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:42.768620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:42.782805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566730eb-db96-42b0-b148-98bb654c0886 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:42.787575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:42.788720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:42.789006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:42.789446 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] PATH_INFO: `/v3/projects/1423919b778544d990acb90eaf504279` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:42.790671 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1423919b778544d990acb90eaf504279)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:42.792529 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:42.839556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1423919b778544d990acb90eaf504279', 'request_id': 'req-72b704e3-9989-4796-ab46-38a25fbad7b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.846166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1423919b778544d990acb90eaf504279', 'request_id': 'req-72b704e3-9989-4796-ab46-38a25fbad7b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.851709 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1423919b778544d990acb90eaf504279', 'request_id': 'req-72b704e3-9989-4796-ab46-38a25fbad7b6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.864438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] The token cache is being invalidate because project 1423919b778544d990acb90eaf504279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:42.864728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.865341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b704e3-9989-4796-ab46-38a25fbad7b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:42.866836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1394/2756] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:42 2022] DELETE /identity/v3/projects/1423919b778544d990acb90eaf504279 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:42.871776 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109421c-fb24-47e2-87f2-88f5d7db063d None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:42.872274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109421c-fb24-47e2-87f2-88f5d7db063d None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:42.872377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109421c-fb24-47e2-87f2-88f5d7db063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:42.891579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-3109421c-fb24-47e2-87f2-88f5d7db063d None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:42.892755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-3109421c-fb24-47e2-87f2-88f5d7db063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:42.949787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1363/2757] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:42.959001 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d1869-ea06-4e45-aabd-a25c76c23a88 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:42.959294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d1869-ea06-4e45-aabd-a25c76c23a88 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:42.959608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d1869-ea06-4e45-aabd-a25c76c23a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:42.979647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c42d1869-ea06-4e45-aabd-a25c76c23a88 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:42.980856 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c42d1869-ea06-4e45-aabd-a25c76c23a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:42.988994 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1395/2758] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:42.993014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109421c-fb24-47e2-87f2-88f5d7db063d None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.002392 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109421c-fb24-47e2-87f2-88f5d7db063d None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.006826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.007406 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.007709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.007977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.008609 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:43.008905 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.010476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.022150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1364/2759] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:10:43.026187 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d1869-ea06-4e45-aabd-a25c76c23a88 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.035746 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d1869-ea06-4e45-aabd-a25c76c23a88 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.040402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.041013 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.041291 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.041560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.042180 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.044142 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.070378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1396/2760] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:43.074470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.084743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf25589-94ce-4c44-8772-a11955311c8c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.139920 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.140522 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.140803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.141103 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.141743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.143677 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.153342 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:43.166273 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1365/2761] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:10:43.170555 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.180937 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ba941-28f8-4b0e-b501-69affb53e557 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.185543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.186625 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.186903 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.187171 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.187838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.189433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.199082 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1397/2762] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:43.203463 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.213151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed20509-43ff-4c27-a8d0-09b5960eeb9c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.217700 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9c01a-d3ec-4357-8490-a6d96683a71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.218288 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9c01a-d3ec-4357-8490-a6d96683a71e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.218567 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9c01a-d3ec-4357-8490-a6d96683a71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.218837 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9c01a-d3ec-4357-8490-a6d96683a71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.219460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9c01a-d3ec-4357-8490-a6d96683a71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.221055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9c01a-d3ec-4357-8490-a6d96683a71e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.231343 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1366/2763] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:43.235284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.245889 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b07429-000d-4e18-9120-6457515ba823 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.250969 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.251631 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.251911 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.252183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] PATH_INFO: `/v3/projects/79e3265e95c14671b85ceb4466927145/users/4cb1893c024b4c549564b3766c4b518a/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.266920 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e3265e95c14671b85ceb4466927145, user_id=4cb1893c024b4c549564b3766c4b518a, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.269547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.282095 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1398/2764] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:43 2022] PUT /identity/v3/projects/79e3265e95c14671b85ceb4466927145/users/4cb1893c024b4c549564b3766c4b518a/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:43.346393 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa51ab6-6012-4077-8b6f-e0a8c49c52cb None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.346678 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa51ab6-6012-4077-8b6f-e0a8c49c52cb None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.346949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa51ab6-6012-4077-8b6f-e0a8c49c52cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.375083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-4fa51ab6-6012-4077-8b6f-e0a8c49c52cb None None] MFA Rules not processed for user `4cb1893c024b4c549564b3766c4b518a`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:43.376327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-4fa51ab6-6012-4077-8b6f-e0a8c49c52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:43.444332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1367/2765] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:43.448802 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.460300 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:43.511407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f8058-cc7a-4f1c-b272-80382b38a67a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.516666 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.517295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.517576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.517848 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.518492 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.520440 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.546393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1399/2766] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:43.550453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa51ab6-6012-4077-8b6f-e0a8c49c52cb None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.561922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa51ab6-6012-4077-8b6f-e0a8c49c52cb None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.617571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.618226 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.618505 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.618774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.619400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.621347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.644388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1368/2767] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:43.648418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.659183 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fad6c3-7e6d-4678-b707-7113df10124c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.663747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.664340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.664616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.664884 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.665512 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.667087 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.678321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1400/2768] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:43.682295 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.692183 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258abbf-30d8-4dbb-8551-1b1329052c1b admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.696813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07918-4ce1-46e1-a873-cd60512cfbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.697405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07918-4ce1-46e1-a873-cd60512cfbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.697684 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07918-4ce1-46e1-a873-cd60512cfbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.697953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07918-4ce1-46e1-a873-cd60512cfbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.698581 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07918-4ce1-46e1-a873-cd60512cfbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.700279 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07918-4ce1-46e1-a873-cd60512cfbd9 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.710553 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1369/2769] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:43.714523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.725267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94768d0-57f2-4339-b4b6-987dda2d8469 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.729980 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.730785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.731128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.731448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] PATH_INFO: `/v3/projects/d0730419435f441aaf2e5a1301c7adbc/users/ec2db595bf7a4dd5a77415098f92b5c8/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.745976 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0730419435f441aaf2e5a1301c7adbc, user_id=ec2db595bf7a4dd5a77415098f92b5c8, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.750101 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.765670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1401/2770] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:43 2022] PUT /identity/v3/projects/d0730419435f441aaf2e5a1301c7adbc/users/ec2db595bf7a4dd5a77415098f92b5c8/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:43.775841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9e947-8c8f-4367-a1fc-a68ff3e7776e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.776129 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9e947-8c8f-4367-a1fc-a68ff3e7776e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.776404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9e947-8c8f-4367-a1fc-a68ff3e7776e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.807468 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-64d9e947-8c8f-4367-a1fc-a68ff3e7776e None None] MFA Rules not processed for user `ec2db595bf7a4dd5a77415098f92b5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:43.808843 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-64d9e947-8c8f-4367-a1fc-a68ff3e7776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:43.850999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:43.875593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1370/2771] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:43.880240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.945118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b97b30-78bd-4ec2-ad4a-068e67e8dff8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:43.950018 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:43.950681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:43.950960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:43.951246 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:43.952536 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:43.954453 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:43.980722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1402/2772] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:43.984774 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9e947-8c8f-4367-a1fc-a68ff3e7776e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:43.995080 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9e947-8c8f-4367-a1fc-a68ff3e7776e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:44.051690 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:44.052292 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:44.052574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:44.052848 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:44.053481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:44.055709 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:44.077303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1371/2773] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:43 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:10:44.081475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:44.091535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991d266-0fee-45e3-b9a9-59a5614e452e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:44.096080 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:44.096669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:44.096946 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:44.097310 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:44.097943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:44.099768 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:44.110188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1403/2774] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:44.114350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:44.124630 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a3811-f36a-4fce-9977-93751fb1b073 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:44.130209 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:44.130861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:44.131146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:44.131424 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] PATH_INFO: `/v3/projects/895fde196c364326a7438b5f3492f9bf/users/95c7cc8ec779485fbbb7e342426340c1/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:44.146798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=895fde196c364326a7438b5f3492f9bf, user_id=95c7cc8ec779485fbbb7e342426340c1, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:44.149445 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:44.161815 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1372/2775] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:44 2022] PUT /identity/v3/projects/895fde196c364326a7438b5f3492f9bf/users/95c7cc8ec779485fbbb7e342426340c1/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:44.165767 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:44.175801 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ff95-71da-4072-815e-ebff35d5b60e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:44.205804 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:44.231783 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5753ab-b92a-4707-a1c3-d296309f1d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:44.232410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5753ab-b92a-4707-a1c3-d296309f1d53 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:44.232622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5753ab-b92a-4707-a1c3-d296309f1d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:44.232893 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5753ab-b92a-4707-a1c3-d296309f1d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:44.233523 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5753ab-b92a-4707-a1c3-d296309f1d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:44.235104 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5753ab-b92a-4707-a1c3-d296309f1d53 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:44.244689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1404/2776] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:44.249318 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:44.259474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ad0fd-2371-465a-96e9-1c075c4a75e7 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:44.264742 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:44.265381 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:44.265658 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:44.265928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] PATH_INFO: `/v3/projects/895fde196c364326a7438b5f3492f9bf/users/95c7cc8ec779485fbbb7e342426340c1/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:44.267861 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=895fde196c364326a7438b5f3492f9bf, user_id=95c7cc8ec779485fbbb7e342426340c1, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:44.270488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:44.280440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1373/2777] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:44 2022] PUT /identity/v3/projects/895fde196c364326a7438b5f3492f9bf/users/95c7cc8ec779485fbbb7e342426340c1/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:44.291294 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:44.291619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:44.291899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:44.320172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] MFA Rules not processed for user `95c7cc8ec779485fbbb7e342426340c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:44.321390 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:44.387956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1405/2778] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:44.399387 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:44.401132 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:44.459858 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171181-2adc-4117-9dbe-10563c68ecb0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:44.463438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:44.464068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:44.464348 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:44.464619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:44.509099 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:44.523190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:44.524651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:44.533368 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1374/2779] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:46.033759 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:46.035424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:46.047457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adc744-bf42-4311-8163-2fc94817da1b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:46.051203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:46.051873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:46.052160 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:46.052432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:46.108016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:46.109457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:46.118375 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1406/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:46.651459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:46.659860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3719cf1-086b-4d85-9352-7c4e68ceec9a service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:46.663408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d7077-0645-480e-b29a-8a044dda9fd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:46.664024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d7077-0645-480e-b29a-8a044dda9fd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:46.664299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d7077-0645-480e-b29a-8a044dda9fd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:46.664573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d7077-0645-480e-b29a-8a044dda9fd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:46.673642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d7077-0645-480e-b29a-8a044dda9fd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:46.675038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d7077-0645-480e-b29a-8a044dda9fd8 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:46.684419 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1375/2781] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:50.542825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:50.596148 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:50.607146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc17b9-458a-47d4-a702-4cf3a53980db service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:50.611828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:50.612458 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:50.612762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:50.613052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] PATH_INFO: `/v3/users/4cb1893c024b4c549564b3766c4b518a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:50.614165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb1893c024b4c549564b3766c4b518a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:50.616179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:50.661907 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb1893c024b4c549564b3766c4b518a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:50.668916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb1893c024b4c549564b3766c4b518a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:50.680890 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb1893c024b4c549564b3766c4b518a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:50.681543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb1893c024b4c549564b3766c4b518a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:50.682179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb1893c024b4c549564b3766c4b518a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:50.695392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1407/2782] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:50 2022] DELETE /identity/v3/users/4cb1893c024b4c549564b3766c4b518a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:50.699805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:50.700319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:50.700427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:50.719411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:50.720625 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:50.777839 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1376/2783] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:50.787642 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:50.789074 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:50.867835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0bdd5-2c5e-40c1-b18e-f6a857eac0db admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:50.871562 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:50.872173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:50.872454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:50.872728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:50.882431 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:50.883916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:50.894194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1408/2784] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:51.242203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:51.243509 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:51.244814 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:51.246573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:51.260954 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf07a41-e187-4d36-9752-bdfcba08c60c service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:51.262164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b4777-34d4-46ed-b11a-c6b1f479acb0 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:51.265861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:51.266654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:51.266941 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:51.267213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:51.267645 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:51.268186 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:51.268541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:51.268753 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] PATH_INFO: `/v3/projects/79e3265e95c14671b85ceb4466927145` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:51.269151 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:51.270247 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e3265e95c14671b85ceb4466927145)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:51.272298 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:51.320816 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e3265e95c14671b85ceb4466927145', 'request_id': 'req-0f5e7f04-3763-4116-827d-9a356c2dda8d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.323377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:51.328023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e3265e95c14671b85ceb4466927145', 'request_id': 'req-0f5e7f04-3763-4116-827d-9a356c2dda8d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.335824 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e3265e95c14671b85ceb4466927145', 'request_id': 'req-0f5e7f04-3763-4116-827d-9a356c2dda8d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.337822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:51.339271 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:51.346362 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] The token cache is being invalidate because project 79e3265e95c14671b85ceb4466927145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:51.346648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.347267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.348935 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1377/2785] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:51 2022] DELETE /identity/v3/projects/79e3265e95c14671b85ceb4466927145 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:51.349091 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1409/2786] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:51.352504 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:51.353862 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:51.417568 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:51.422586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e7f04-3763-4116-827d-9a356c2dda8d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:51.428968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:51.429899 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:51.430243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:51.430533 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] PATH_INFO: `/v3/users/ec2db595bf7a4dd5a77415098f92b5c8` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:51.431985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2db595bf7a4dd5a77415098f92b5c8)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:51.434308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:51.480260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2db595bf7a4dd5a77415098f92b5c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.481012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2db595bf7a4dd5a77415098f92b5c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.488488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2db595bf7a4dd5a77415098f92b5c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.500511 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2db595bf7a4dd5a77415098f92b5c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.501300 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2db595bf7a4dd5a77415098f92b5c8'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:51.515061 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1378/2787] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:51 2022] DELETE /identity/v3/users/ec2db595bf7a4dd5a77415098f92b5c8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:10:51.885964 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:51.896761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45e055-6798-4a9a-9477-cbe1036707e7 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:51.964185 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:51.964818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:51.965093 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:51.965361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] PATH_INFO: `/v3/projects/d0730419435f441aaf2e5a1301c7adbc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:51.966496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0730419435f441aaf2e5a1301c7adbc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:51.968970 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:52.030005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0730419435f441aaf2e5a1301c7adbc', 'request_id': 'req-7c2a3469-f704-4db0-b882-510b15fe9353'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.036702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0730419435f441aaf2e5a1301c7adbc', 'request_id': 'req-7c2a3469-f704-4db0-b882-510b15fe9353'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.042583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0730419435f441aaf2e5a1301c7adbc', 'request_id': 'req-7c2a3469-f704-4db0-b882-510b15fe9353'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.054565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] The token cache is being invalidate because project d0730419435f441aaf2e5a1301c7adbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:52.054859 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.055515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.057010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1410/2788] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:51 2022] DELETE /identity/v3/projects/d0730419435f441aaf2e5a1301c7adbc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:52.061208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:52.062829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:52.076582 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ed425-3d39-4d2a-b73d-96097af7b3ab admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:52.134670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:52.135219 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:52.135512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:52.135804 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] PATH_INFO: `/v3/users/95c7cc8ec779485fbbb7e342426340c1` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:52.136962 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c7cc8ec779485fbbb7e342426340c1)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:52.139408 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:52.187104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c7cc8ec779485fbbb7e342426340c1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.187797 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c7cc8ec779485fbbb7e342426340c1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.194971 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c7cc8ec779485fbbb7e342426340c1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.208135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c7cc8ec779485fbbb7e342426340c1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.208831 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-da0e8dfc-edb7-447a-ad55-71a8c28bcd06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c7cc8ec779485fbbb7e342426340c1'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.212423 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:52.223270 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1379/2789] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:10:52 2022] DELETE /identity/v3/users/95c7cc8ec779485fbbb7e342426340c1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:52.519038 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:52.520735 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:52.543933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3469-f704-4db0-b882-510b15fe9353 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:52.605327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:52.605974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:52.606251 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:52.606518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] PATH_INFO: `/v3/projects/895fde196c364326a7438b5f3492f9bf` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:52.608260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=895fde196c364326a7438b5f3492f9bf)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:52.610177 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:52.634769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:52.660548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895fde196c364326a7438b5f3492f9bf', 'request_id': 'req-29039eca-e18e-4d5f-8d66-d217dcc1da67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.666953 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895fde196c364326a7438b5f3492f9bf', 'request_id': 'req-29039eca-e18e-4d5f-8d66-d217dcc1da67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.673361 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895fde196c364326a7438b5f3492f9bf', 'request_id': 'req-29039eca-e18e-4d5f-8d66-d217dcc1da67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.687423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] The token cache is being invalidate because project 895fde196c364326a7438b5f3492f9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:10:52.687742 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.688400 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-29039eca-e18e-4d5f-8d66-d217dcc1da67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:10:52.689899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1411/2790] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:10:52 2022] DELETE /identity/v3/projects/895fde196c364326a7438b5f3492f9bf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:52.694969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd80e62-1e61-4a94-a05a-fb3eb0157f8c None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:52.695252 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd80e62-1e61-4a94-a05a-fb3eb0157f8c None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:52.695544 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd80e62-1e61-4a94-a05a-fb3eb0157f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:52.715222 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-5fd80e62-1e61-4a94-a05a-fb3eb0157f8c None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:52.716476 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-5fd80e62-1e61-4a94-a05a-fb3eb0157f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:52.781693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1380/2791] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:52.789364 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f0f5-414f-452c-9466-23eb5a595dab None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:52.789656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f0f5-414f-452c-9466-23eb5a595dab None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:52.791237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f0f5-414f-452c-9466-23eb5a595dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:52.812105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0492f0f5-414f-452c-9466-23eb5a595dab None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:52.813298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0492f0f5-414f-452c-9466-23eb5a595dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:52.821399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1412/2792] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:52.825784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd80e62-1e61-4a94-a05a-fb3eb0157f8c None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:52.836733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd80e62-1e61-4a94-a05a-fb3eb0157f8c None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:52.842529 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:52.843206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:52.843584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:52.843922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:52.844675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:10:52.845035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:52.846682 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:52.859155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1381/2793] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:10:52.863636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492f0f5-414f-452c-9466-23eb5a595dab None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:52.874259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492f0f5-414f-452c-9466-23eb5a595dab None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:52.878967 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:52.879592 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:52.879871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:52.880139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:52.880773 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:52.882655 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:52.911728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1413/2794] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:10:52 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:10:52.916018 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:52.926556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8656ef-57c3-4a10-b4c4-f45cd2a535e8 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:52.983811 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:52.984418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:52.984689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:52.984968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:52.985600 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:52.987591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:53.009005 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1382/2795] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:10:52 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:10:53.013457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:53.023873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d3d9-730b-4d4e-89af-df1e7d75d1df admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:53.028464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:53.029559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:53.029822 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:53.030090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:53.030745 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:53.032357 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:53.042366 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1414/2796] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:53.047436 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:53.058085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57004e3d-37e2-4ace-bfd8-1e88fa129f43 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:53.062803 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be62e3-96fc-4244-8b7f-749039f2332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:53.063543 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be62e3-96fc-4244-8b7f-749039f2332c admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:53.063860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be62e3-96fc-4244-8b7f-749039f2332c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:53.064162 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be62e3-96fc-4244-8b7f-749039f2332c admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:53.064813 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be62e3-96fc-4244-8b7f-749039f2332c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:53.066428 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be62e3-96fc-4244-8b7f-749039f2332c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:53.077883 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1383/2797] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:10:53.081811 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:53.092427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc63ed-c58b-484f-a432-85b5d8b84bba admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:53.097149 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:53.097858 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:53.098137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:53.098408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] PATH_INFO: `/v3/projects/460da1ec0c6a40eeb0e3760d1ee7b33f/users/3d213252fb6a49ed9e026365f7945509/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:53.115006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460da1ec0c6a40eeb0e3760d1ee7b33f, user_id=3d213252fb6a49ed9e026365f7945509, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:53.117682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:53.131544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1415/2798] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:10:53 2022] PUT /identity/v3/projects/460da1ec0c6a40eeb0e3760d1ee7b33f/users/3d213252fb6a49ed9e026365f7945509/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:10:53.136789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb86db-86cb-45f3-89d0-682d9f532546 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:53.137198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb86db-86cb-45f3-89d0-682d9f532546 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:53.137618 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb86db-86cb-45f3-89d0-682d9f532546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:53.156942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1aeb86db-86cb-45f3-89d0-682d9f532546 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:53.158135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1aeb86db-86cb-45f3-89d0-682d9f532546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:53.221459 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1384/2799] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:10:53.231578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:53.233968 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:53.303324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e0a3-c600-4bcb-8131-b9e5f64ba7b9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:53.307240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:53.307899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:53.308178 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:53.308454 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:53.319192 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:53.320808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:53.332212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1416/2800] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:54.251322 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d427e-f339-402a-bbef-05c64c3c65d5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:54.251853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d427e-f339-402a-bbef-05c64c3c65d5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:54.251965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d427e-f339-402a-bbef-05c64c3c65d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:54.260488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:54.286893 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-463d427e-f339-402a-bbef-05c64c3c65d5 None None] MFA Rules not processed for user `3d213252fb6a49ed9e026365f7945509`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:54.288192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-463d427e-f339-402a-bbef-05c64c3c65d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:54.372093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1385/2801] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:54.380762 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:54.382311 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:54.455408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:10:54.480691 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169372c4-088f-4955-a4e9-9136e7c6e486 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:54.484365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:54.484974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:54.485255 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:54.485528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:54.493819 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:54.495173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:54.505341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1417/2802] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:10:55.865612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c904f4-3b4a-4059-92ff-019a8e779687 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:55.866122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c904f4-3b4a-4059-92ff-019a8e779687 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:55.866122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c904f4-3b4a-4059-92ff-019a8e779687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:55.885131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-f5c904f4-3b4a-4059-92ff-019a8e779687 None None] MFA Rules not processed for user `3d213252fb6a49ed9e026365f7945509`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:10:55.886323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f5c904f4-3b4a-4059-92ff-019a8e779687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:55.893094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1386/2803] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:10:56.932462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:10:56.933875 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:57.023808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d11ac-9cfa-46f3-bea6-7aeb711bfbba service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:10:57.028791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c43ad-d65e-441f-847e-c4c23ea51774 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:10:57.029417 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c43ad-d65e-441f-847e-c4c23ea51774 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:10:57.029695 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c43ad-d65e-441f-847e-c4c23ea51774 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:10:57.029966 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c43ad-d65e-441f-847e-c4c23ea51774 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:10:57.031702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fd2c43ad-d65e-441f-847e-c4c23ea51774 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:10:57.042843 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c43ad-d65e-441f-847e-c4c23ea51774 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:10:57.044436 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c43ad-d65e-441f-847e-c4c23ea51774 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:10:57.054933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1418/2804] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:10:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:11:05.879834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=34 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:11:07.050407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=54 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:11:11.218055 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:11.218603 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:11.218707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:11.237767 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] MFA Rules not processed for user `3d213252fb6a49ed9e026365f7945509`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:11:11.238961 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:11.245660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1387/2805] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:11:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:11:21.231922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:11:32.785662 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279b284-792a-439c-a797-aba94c6e56ef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:32.786176 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279b284-792a-439c-a797-aba94c6e56ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:32.786287 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279b284-792a-439c-a797-aba94c6e56ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:32.815569 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6279b284-792a-439c-a797-aba94c6e56ef None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:11:32.816697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6279b284-792a-439c-a797-aba94c6e56ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:32.877779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1419/2806] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 23:11:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:11:32.923317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:32.925229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:32.994334 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675771-e137-4062-ac0b-be58d5f6c682 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:32.999323 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:33.000008 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:33.000287 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:33.000591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:33.009525 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:33.011101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:33.018997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1388/2807] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:11:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:11:42.871347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=32 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:11:43.016000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=32 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:11:49.077552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348f1e7-61c4-45af-8c87-b45839c14a72 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:49.078008 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348f1e7-61c4-45af-8c87-b45839c14a72 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:49.078105 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348f1e7-61c4-45af-8c87-b45839c14a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:49.097331 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6348f1e7-61c4-45af-8c87-b45839c14a72 None None] MFA Rules not processed for user `452fcf1ae7a34bb1a2c1010575de2baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:11:49.098510 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6348f1e7-61c4-45af-8c87-b45839c14a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:49.105438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1420/2808] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 23:11:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:11:49.117356 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:49.128472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40cb07-9c89-4c39-a9bb-ee4a1b8c63d7 service swift] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:49.133407 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ceb38886d34f9d8e59b2871989446c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:49.133991 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:49.134269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:49.134541 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:49.142909 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:49.144499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:49.152417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1389/2809] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 23:11:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:11:51.035480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348f1e7-61c4-45af-8c87-b45839c14a72 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.048535 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348f1e7-61c4-45af-8c87-b45839c14a72 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.053131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.053761 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.054036 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.054303 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] PATH_INFO: `/v3/users/3d213252fb6a49ed9e026365f7945509` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.055732 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d213252fb6a49ed9e026365f7945509)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.057636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.102675 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d213252fb6a49ed9e026365f7945509'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.109556 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d213252fb6a49ed9e026365f7945509'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.121871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d213252fb6a49ed9e026365f7945509'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.122526 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d213252fb6a49ed9e026365f7945509'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.123127 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bdbe1c4c-5e5b-4131-994a-e405dc655471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d213252fb6a49ed9e026365f7945509'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.136484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1421/2810] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:11:51 2022] DELETE /identity/v3/users/3d213252fb6a49ed9e026365f7945509 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:11:51.304274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.314988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9d316-d25f-42df-8fa6-803b6a340d95 service swift] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.373294 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.373853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.374131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.374401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] PATH_INFO: `/v3/projects/460da1ec0c6a40eeb0e3760d1ee7b33f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.376093 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=460da1ec0c6a40eeb0e3760d1ee7b33f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.377976 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.425048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460da1ec0c6a40eeb0e3760d1ee7b33f', 'request_id': 'req-faebf1c8-e9de-4eb9-817f-098b0379d559'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.431332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460da1ec0c6a40eeb0e3760d1ee7b33f', 'request_id': 'req-faebf1c8-e9de-4eb9-817f-098b0379d559'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.439152 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460da1ec0c6a40eeb0e3760d1ee7b33f', 'request_id': 'req-faebf1c8-e9de-4eb9-817f-098b0379d559'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.449706 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] The token cache is being invalidate because project 460da1ec0c6a40eeb0e3760d1ee7b33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:11:51.449997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.450613 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-faebf1c8-e9de-4eb9-817f-098b0379d559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:11:51.452105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1390/2811] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:11:51 2022] DELETE /identity/v3/projects/460da1ec0c6a40eeb0e3760d1ee7b33f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:11:51.456928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c188-e250-4c04-a755-71081fa530d3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.457209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c188-e250-4c04-a755-71081fa530d3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.457476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c188-e250-4c04-a755-71081fa530d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.477274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3da9c188-e250-4c04-a755-71081fa530d3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:11:51.478479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3da9c188-e250-4c04-a755-71081fa530d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:51.536021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1422/2812] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:11:51.543748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0438d-7704-4197-ab25-9b1e4e09742e None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.544034 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0438d-7704-4197-ab25-9b1e4e09742e None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.544304 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0438d-7704-4197-ab25-9b1e4e09742e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.564266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-80b0438d-7704-4197-ab25-9b1e4e09742e None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:11:51.565460 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-80b0438d-7704-4197-ab25-9b1e4e09742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:51.573164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1391/2813] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:11:51.577321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9c188-e250-4c04-a755-71081fa530d3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.586684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9c188-e250-4c04-a755-71081fa530d3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.591430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.592129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.592410 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.592681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.593324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:11:51.593624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.595835 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.608020 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1423/2814] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:11:51.613211 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0438d-7704-4197-ab25-9b1e4e09742e None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.622752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0438d-7704-4197-ab25-9b1e4e09742e None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.627435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.628003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.628278 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.628545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.629166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.631030 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.656820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1392/2815] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:11:51 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:11:51.661574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.671693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdff28e-cbc3-46f8-921e-10aeaaadc8d6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.722085 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:11:51.732586 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.733437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.733437 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.733718 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.734347 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.736320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.758281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1424/2816] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:11:51 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:11:51.762414 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.773066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e0a9b-5e09-4f42-a58d-6de29693069f admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.777636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.778229 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.778508 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.778781 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.779413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.781016 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.791266 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1393/2817] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:11:51.795049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.805245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efdd2f-b78f-441d-8f02-086ad3e653a9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.809829 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5f4af-8f64-4d92-a9f3-27eed2701342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.810408 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5f4af-8f64-4d92-a9f3-27eed2701342 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.810688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5f4af-8f64-4d92-a9f3-27eed2701342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.810956 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5f4af-8f64-4d92-a9f3-27eed2701342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.811624 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5f4af-8f64-4d92-a9f3-27eed2701342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.813184 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5f4af-8f64-4d92-a9f3-27eed2701342 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.823457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1425/2818] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:11:51.827401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.837931 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6812f-d405-44ba-86ea-a6dda2e3ed9d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:51.842761 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:51.843421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.843734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.844006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] PATH_INFO: `/v3/projects/2dfee51e14ba45b5bad8bc5614eaf180/users/846501e56205419397441be21eba85e2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.859345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfee51e14ba45b5bad8bc5614eaf180, user_id=846501e56205419397441be21eba85e2, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:51.862009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:51.874394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1394/2819] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:11:51 2022] PUT /identity/v3/projects/2dfee51e14ba45b5bad8bc5614eaf180/users/846501e56205419397441be21eba85e2/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:11:51.884393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197770-d443-4fc5-8471-abffaad2f481 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:51.884619 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197770-d443-4fc5-8471-abffaad2f481 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:51.884891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197770-d443-4fc5-8471-abffaad2f481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:51.912728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-28197770-d443-4fc5-8471-abffaad2f481 None None] MFA Rules not processed for user `846501e56205419397441be21eba85e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:11:51.913928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-28197770-d443-4fc5-8471-abffaad2f481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:51.974862 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1426/2820] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:11:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:11:51.986472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:11:51.988178 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:11:52.003079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:11:52.047710 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb15cd-8c99-4971-bfce-3ee348fdc75c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:11:52.051262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bc9ad-d48b-416a-a116-5e549bd5bd1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cf808a1f5dd45bea0a3a81131532b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:11:52.051891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bc9ad-d48b-416a-a116-5e549bd5bd1c service glance] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:11:52.052166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bc9ad-d48b-416a-a116-5e549bd5bd1c service glance] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:11:52.052433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bc9ad-d48b-416a-a116-5e549bd5bd1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:11:52.060754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bc9ad-d48b-416a-a116-5e549bd5bd1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:11:52.062159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bc9ad-d48b-416a-a116-5e549bd5bd1c service glance] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:11:52.071164 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1395/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:11:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:12:01.972214 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:02.067306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=20 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:18.761620 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28197770-d443-4fc5-8471-abffaad2f481 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:18.823133 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28197770-d443-4fc5-8471-abffaad2f481 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:18.827643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:18.828268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:18.828546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:18.828818 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] PATH_INFO: `/v3/users/846501e56205419397441be21eba85e2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:18.829939 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=846501e56205419397441be21eba85e2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:18.831886 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:18.876173 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '846501e56205419397441be21eba85e2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:18.883833 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '846501e56205419397441be21eba85e2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:18.895693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '846501e56205419397441be21eba85e2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:18.896356 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '846501e56205419397441be21eba85e2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:18.896950 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '846501e56205419397441be21eba85e2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:18.909138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1427/2822] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:12:18 2022] DELETE /identity/v3/users/846501e56205419397441be21eba85e2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:12:18.913623 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:18.914108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:18.914214 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:18.933619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:12:18.934817 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:18.993888 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1396/2823] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:12:19.004053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:19.005547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:19.061869 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab9fd6-9d6a-4ecc-ba31-84046514a6a2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:19.065473 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:19.066064 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:19.066340 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:19.066608 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:19.075244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:19.076694 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:19.085882 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1428/2824] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:12:19.350889 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:19.352444 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:19.365685 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156f997-f744-407f-b6f4-2387ac8898f3 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:19.370643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:19.371274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:19.371592 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:19.371868 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] PATH_INFO: `/v3/projects/2dfee51e14ba45b5bad8bc5614eaf180` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:19.372997 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfee51e14ba45b5bad8bc5614eaf180)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:19.374950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:19.427108 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfee51e14ba45b5bad8bc5614eaf180', 'request_id': 'req-8aa23c19-ae10-4a3f-a97f-62b632af0d99'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:19.433473 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfee51e14ba45b5bad8bc5614eaf180', 'request_id': 'req-8aa23c19-ae10-4a3f-a97f-62b632af0d99'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:19.440442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfee51e14ba45b5bad8bc5614eaf180', 'request_id': 'req-8aa23c19-ae10-4a3f-a97f-62b632af0d99'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:19.450697 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] The token cache is being invalidate because project 2dfee51e14ba45b5bad8bc5614eaf180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:12:19.450985 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:19.451787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:19.453267 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1397/2825] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:12:19 2022] DELETE /identity/v3/projects/2dfee51e14ba45b5bad8bc5614eaf180 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:12:23.932918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:23.941846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae85d7-105a-44a2-8c7d-5614c9053da0 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:23.960341 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:23.994528 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:23.995128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:23.995403 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:23.995697 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:23.997559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:24.079622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:24.081560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:24.102433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1429/2826] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:12:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:12:25.287805 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:25.289313 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:25.302828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa23c19-ae10-4a3f-a97f-62b632af0d99 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:25.306670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:25.307269 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:25.308036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:25.308316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:25.310114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:25.324023 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:25.325647 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:25.339693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1398/2827] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:12:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:12:34.099834 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=53 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:35.331832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:44.427169 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:44.428703 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:44.440636 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dbdfb-ad38-4bbe-ae48-bde369d1fe18 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:44.444328 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:44.444916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:44.445193 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:44.445462 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:44.447123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:44.508825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:44.510257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:44.519845 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1430/2828] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:12:50.662882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:50.665084 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:50.681950 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ceaf9-9c20-4108-ade2-bcf393a0e0c6 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:50.686780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:50.687418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:50.687734 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:50.688013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] PATH_INFO: `/v3/users/3cbefab6dddb45c381158126d203f66b` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:50.689221 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cbefab6dddb45c381158126d203f66b)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:50.691148 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:50.744404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cbefab6dddb45c381158126d203f66b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:50.745128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cbefab6dddb45c381158126d203f66b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:50.752358 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cbefab6dddb45c381158126d203f66b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:50.764447 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cbefab6dddb45c381158126d203f66b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:50.765097 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cbefab6dddb45c381158126d203f66b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:50.777142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1399/2829] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:12:50 2022] DELETE /identity/v3/users/3cbefab6dddb45c381158126d203f66b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:12:50.948693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:50.959913 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cac416-79b2-4a19-81ad-ef6dc2a80f94 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:51.015413 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:51.016071 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:51.016352 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:51.016623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] PATH_INFO: `/v3/projects/82213ebf52f24af49758dd17e5e1d129` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:51.018267 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=82213ebf52f24af49758dd17e5e1d129)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:51.020220 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:51.066593 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82213ebf52f24af49758dd17e5e1d129', 'request_id': 'req-308f14e1-2fca-484a-99ab-aa92a41a64ba'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.072506 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82213ebf52f24af49758dd17e5e1d129', 'request_id': 'req-308f14e1-2fca-484a-99ab-aa92a41a64ba'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.077928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82213ebf52f24af49758dd17e5e1d129', 'request_id': 'req-308f14e1-2fca-484a-99ab-aa92a41a64ba'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.090002 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] The token cache is being invalidate because project 82213ebf52f24af49758dd17e5e1d129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:12:51.090289 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.090916 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.092411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1431/2830] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:12:50 2022] DELETE /identity/v3/projects/82213ebf52f24af49758dd17e5e1d129 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:12:51.096984 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:51.098371 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:51.111138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1defe-89b9-4a21-b371-6122ea729c8c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:51.165928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:51.166478 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:51.166755 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:51.167024 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] PATH_INFO: `/v3/users/96db37066a8b4e3ba2fed7caaee50398` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:51.168138 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96db37066a8b4e3ba2fed7caaee50398)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:51.170027 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:51.214947 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96db37066a8b4e3ba2fed7caaee50398'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.215705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96db37066a8b4e3ba2fed7caaee50398'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.223026 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96db37066a8b4e3ba2fed7caaee50398'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.234616 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96db37066a8b4e3ba2fed7caaee50398'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.235245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96db37066a8b4e3ba2fed7caaee50398'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.236908 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:51.249604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1400/2831] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:12:51 2022] DELETE /identity/v3/users/96db37066a8b4e3ba2fed7caaee50398 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:12:51.422322 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:51.423936 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:51.437891 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f14e1-2fca-484a-99ab-aa92a41a64ba admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:51.460461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:51.502808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:51.503451 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:51.503796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:51.504052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] PATH_INFO: `/v3/projects/bddf9b8f9d454fd789a20e927a5692f9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:51.505248 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bddf9b8f9d454fd789a20e927a5692f9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:51.507146 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:51.558574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bddf9b8f9d454fd789a20e927a5692f9', 'request_id': 'req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.565689 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bddf9b8f9d454fd789a20e927a5692f9', 'request_id': 'req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.573568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bddf9b8f9d454fd789a20e927a5692f9', 'request_id': 'req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.590831 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] The token cache is being invalidate because project bddf9b8f9d454fd789a20e927a5692f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:12:51.591537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.591726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:12:51.593210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1432/2832] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:12:51 2022] DELETE /identity/v3/projects/bddf9b8f9d454fd789a20e927a5692f9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:13:01.247979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:01.589083 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:13.404374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:13.405853 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:13.462724 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e03fd2-98b4-421b-ac34-d174b7332343 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:13.511413 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:13.512064 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:13.512347 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:13.512622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:13.514411 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:13.523586 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:13.524999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:13.532890 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1401/2833] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:17.695610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:17.697228 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:17.709754 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ca3c-d5c6-4e3b-aec4-5736ffba343c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:17.713418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:17.714155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:17.714432 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:17.714701 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:17.716578 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:17.727570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:17.729043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:17.738230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1433/2834] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:17.810789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:17.819421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ace3-67bf-4c46-b827-cc13bb011570 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:17.823539 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:17.824151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:17.824431 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:17.824703 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:17.826432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:17.896762 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:17.898174 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:17.909573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1402/2835] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:17.972704 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:17.974068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:18.043230 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b8588-4847-476c-8185-428e6c0930ec service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:18.048333 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d136cd-b233-4b96-8d83-18b005bf4372 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:18.048931 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d136cd-b233-4b96-8d83-18b005bf4372 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:18.049205 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d136cd-b233-4b96-8d83-18b005bf4372 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:18.049476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d136cd-b233-4b96-8d83-18b005bf4372 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:18.051170 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d5d136cd-b233-4b96-8d83-18b005bf4372 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:18.062434 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d136cd-b233-4b96-8d83-18b005bf4372 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:18.064058 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d136cd-b233-4b96-8d83-18b005bf4372 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:18.073658 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1434/2836] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:23.027481 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:23.028938 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:23.042595 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a1657-b2d5-4df2-b4ee-99a7f0e5630a service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:23.048296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:23.048937 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:23.049216 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:23.049488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:23.051260 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:23.065206 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:23.066798 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:23.079728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1403/2837] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:26.319447 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f0dc4-74b2-47d6-b47d-1679326f561d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:26.319986 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f0dc4-74b2-47d6-b47d-1679326f561d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:26.320096 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f0dc4-74b2-47d6-b47d-1679326f561d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:26.338637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-389f0dc4-74b2-47d6-b47d-1679326f561d None None] MFA Rules not processed for user `26bf037a0dd84ca9b46d8e2693de8bc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:26.339865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-389f0dc4-74b2-47d6-b47d-1679326f561d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:26.393206 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1435/2838] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:13:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:26.694499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:26.703715 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888b58-8a63-4c88-8054-61e7cb2eb697 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:26.707249 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:26.707875 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:26.708160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:26.708437 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:26.710151 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:26.720480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:26.721941 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:26.730035 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1404/2839] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:31.244370 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f0dc4-74b2-47d6-b47d-1679326f561d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:31.258963 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f0dc4-74b2-47d6-b47d-1679326f561d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:31.264443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:31.265047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:31.265324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:31.266247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:31.268011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:31.281075 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:31.283438 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:31.294600 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1436/2840] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:36.726488 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=92 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:41.290799 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=69 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:46.515173 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:46.517115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:46.587101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ace3-295c-41eb-ba44-f9e1972d7142 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:46.591750 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:46.592377 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:46.592656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:46.592929 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] PATH_INFO: `/v3/users/26bf037a0dd84ca9b46d8e2693de8bc4` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:46.594065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bf037a0dd84ca9b46d8e2693de8bc4)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:46.595993 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:46.640564 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bf037a0dd84ca9b46d8e2693de8bc4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.641205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bf037a0dd84ca9b46d8e2693de8bc4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.648983 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bf037a0dd84ca9b46d8e2693de8bc4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.661045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bf037a0dd84ca9b46d8e2693de8bc4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.661635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-36e88b52-7ffe-4016-974b-670a50e1674c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bf037a0dd84ca9b46d8e2693de8bc4'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.674579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1405/2841] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:14:46 2022] DELETE /identity/v3/users/26bf037a0dd84ca9b46d8e2693de8bc4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:46.848808 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:46.862053 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b68d17-41a4-42e1-b2e3-65e081485c3a service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:46.918912 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:46.919475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:46.919794 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:46.920066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] PATH_INFO: `/v3/projects/f319bdd0037244749fb635d82729b782` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:46.921195 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f319bdd0037244749fb635d82729b782)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:46.923079 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:46.968559 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f319bdd0037244749fb635d82729b782', 'request_id': 'req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.974900 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f319bdd0037244749fb635d82729b782', 'request_id': 'req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.980407 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f319bdd0037244749fb635d82729b782', 'request_id': 'req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.992789 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] The token cache is being invalidate because project f319bdd0037244749fb635d82729b782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:14:46.993081 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.993698 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-89fd67d1-432c-4345-9bf6-2e1a8b5e81f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:46.995189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1437/2842] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:14:46 2022] DELETE /identity/v3/projects/f319bdd0037244749fb635d82729b782 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:14:47.000253 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6de5-7a46-42bb-9d44-e38d642904d6 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.000540 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6de5-7a46-42bb-9d44-e38d642904d6 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.000808 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6de5-7a46-42bb-9d44-e38d642904d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.020397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-55eb6de5-7a46-42bb-9d44-e38d642904d6 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:47.021588 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-55eb6de5-7a46-42bb-9d44-e38d642904d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:47.078299 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1406/2843] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:47.087570 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582e50-1205-483e-930b-2af568857770 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.087857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582e50-1205-483e-930b-2af568857770 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.088130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582e50-1205-483e-930b-2af568857770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.108583 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-07582e50-1205-483e-930b-2af568857770 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:47.109798 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-07582e50-1205-483e-930b-2af568857770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:47.118190 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1438/2844] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:47.122451 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb6de5-7a46-42bb-9d44-e38d642904d6 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.132923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb6de5-7a46-42bb-9d44-e38d642904d6 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.138651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.139274 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.139591 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.139881 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.140524 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:14:47.140828 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.142435 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.158350 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1407/2845] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:14:47.162727 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07582e50-1205-483e-930b-2af568857770 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.172189 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07582e50-1205-483e-930b-2af568857770 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.177313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.177921 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.178197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.178468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.179097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.181313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.222319 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1439/2846] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:14:47 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:14:47.226277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.236366 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e3544-2dc9-4acb-944f-4746496e734e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.240660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:47.300809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.301421 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.301705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.301977 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.302620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.304562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.329072 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1408/2847] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:14:47 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:14:47.333320 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.343878 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9799214-61ad-40c1-8d9b-ccf35a350c8c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.348681 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.349339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.349627 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.349896 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.350567 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.352208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.362823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1440/2848] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:14:47.366806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.377073 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ce625-7c6c-4d29-85c9-1aa44c8edb29 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.382337 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73409b5c-9674-43db-86e5-5470467873ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.382932 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73409b5c-9674-43db-86e5-5470467873ef admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.383245 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73409b5c-9674-43db-86e5-5470467873ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.383554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73409b5c-9674-43db-86e5-5470467873ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.384190 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73409b5c-9674-43db-86e5-5470467873ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.385780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73409b5c-9674-43db-86e5-5470467873ef admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.398310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1409/2849] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:14:47.402489 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.413203 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130e0b9-abeb-40e4-a45b-d0991901e648 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.417665 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.418317 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.418595 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.418876 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] PATH_INFO: `/v3/projects/e679e7afe9de4010acc09828720aec62/users/5126451e8e654a78ba9663ee70ef4183/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.434769 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e679e7afe9de4010acc09828720aec62, user_id=5126451e8e654a78ba9663ee70ef4183, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.437379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.452684 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1441/2850] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:14:47 2022] PUT /identity/v3/projects/e679e7afe9de4010acc09828720aec62/users/5126451e8e654a78ba9663ee70ef4183/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:47.464353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c48bfc-b35b-4ba9-b0ee-17d585f02be5 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.464643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c48bfc-b35b-4ba9-b0ee-17d585f02be5 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.464917 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c48bfc-b35b-4ba9-b0ee-17d585f02be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.493302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-49c48bfc-b35b-4ba9-b0ee-17d585f02be5 None None] MFA Rules not processed for user `5126451e8e654a78ba9663ee70ef4183`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:47.494669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-49c48bfc-b35b-4ba9-b0ee-17d585f02be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:47.591965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1410/2851] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:47.593120 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.616392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:47.670383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73598b1c-f742-465d-96e2-be44e11e21e0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.675736 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.676463 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.676807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.677138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.677861 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.679928 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.713544 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1442/2852] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:14:47 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:14:47.719122 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c48bfc-b35b-4ba9-b0ee-17d585f02be5 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.731388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c48bfc-b35b-4ba9-b0ee-17d585f02be5 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.793662 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.794275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.794554 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.794821 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.795442 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.797372 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.823792 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1411/2853] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:14:47 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:14:47.825327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.836321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56e370-7039-40ce-8b79-9547317d7063 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.841482 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.842179 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.842452 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.842722 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.843604 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.845374 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.856345 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1443/2854] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:14:47.860847 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.867988 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:47.872562 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e036ec-daa2-4cd2-9030-9a202a6fbe64 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:47.879364 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:47.880109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.880384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.880651 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] PATH_INFO: `/v3/projects/0b849b3f19a543eca4b8b9c85de90d9f/users/d0aa28b7705a418cac920c95a025ce5d/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.895835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b849b3f19a543eca4b8b9c85de90d9f, user_id=d0aa28b7705a418cac920c95a025ce5d, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:47.898493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:47.913009 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1412/2855] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:14:47 2022] PUT /identity/v3/projects/0b849b3f19a543eca4b8b9c85de90d9f/users/d0aa28b7705a418cac920c95a025ce5d/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:47.917099 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.927641 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f37048-72ce-41c8-82b7-07325f9cfa4a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:48.004319 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:48.006159 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:48.006335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:48.007054 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:48.007404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:48.007775 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:48.008500 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:48.012305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:48.024021 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1444/2856] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:14:48.028399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:48.040245 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c3914-4b4d-43be-ad79-be140df3aaa1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:48.045846 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df9c58-5f4e-4928-b472-fd3f85e966a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:48.046803 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df9c58-5f4e-4928-b472-fd3f85e966a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:48.047343 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df9c58-5f4e-4928-b472-fd3f85e966a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:48.048033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df9c58-5f4e-4928-b472-fd3f85e966a5 admin admin] PATH_INFO: `/v3/projects/0b849b3f19a543eca4b8b9c85de90d9f/users/d0aa28b7705a418cac920c95a025ce5d/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:48.051011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df9c58-5f4e-4928-b472-fd3f85e966a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b849b3f19a543eca4b8b9c85de90d9f, user_id=d0aa28b7705a418cac920c95a025ce5d, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:48.054423 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df9c58-5f4e-4928-b472-fd3f85e966a5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:48.077247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1445/2857] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:14:48 2022] PUT /identity/v3/projects/0b849b3f19a543eca4b8b9c85de90d9f/users/d0aa28b7705a418cac920c95a025ce5d/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:48.079707 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc75b-60d7-4e1f-b614-70a8852210cc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:48.080275 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc75b-60d7-4e1f-b614-70a8852210cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:48.081063 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc75b-60d7-4e1f-b614-70a8852210cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:48.106135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852e7bf-08c4-4d1a-9d8b-379afd2ee3b6 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:48.118823 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5e9fc75b-60d7-4e1f-b614-70a8852210cc None None] MFA Rules not processed for user `d0aa28b7705a418cac920c95a025ce5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:48.120688 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5e9fc75b-60d7-4e1f-b614-70a8852210cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:48.196745 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786e4e7b1c5147e093693767b229e52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:48.197675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:48.198246 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:48.198526 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:48.201025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:48.218015 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:48.219654 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:48.229888 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:48.235576 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1413/2858] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:14:48.239790 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1446/2859] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:48.253918 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:48.255636 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:48.339344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c37273-fbc7-4c45-8008-4aaf4b8731e6 service nova] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:48.343087 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:48.343744 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:48.344028 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:48.344303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:48.354231 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:48.355691 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:48.370763 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1414/2860] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:48.452962 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc75b-60d7-4e1f-b614-70a8852210cc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:48.462839 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc75b-60d7-4e1f-b614-70a8852210cc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:48.466418 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661b301-dca1-4d3b-b433-7e6659c352e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:48.467022 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b301-dca1-4d3b-b433-7e6659c352e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:48.467422 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b301-dca1-4d3b-b433-7e6659c352e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:48.467753 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b301-dca1-4d3b-b433-7e6659c352e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:48.526049 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661b301-dca1-4d3b-b433-7e6659c352e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:48.527484 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661b301-dca1-4d3b-b433-7e6659c352e0 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:48.537132 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1447/2861] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:57.575068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:57.603723 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:57.689177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791db0e-b5f4-47e6-970c-3704041b1404 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:57.694160 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:57.694612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:57.694960 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:57.695302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] PATH_INFO: `/v3/users/5126451e8e654a78ba9663ee70ef4183` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:57.696556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5126451e8e654a78ba9663ee70ef4183)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:57.698534 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:57.750607 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5126451e8e654a78ba9663ee70ef4183'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:57.751303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5126451e8e654a78ba9663ee70ef4183'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:57.758834 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5126451e8e654a78ba9663ee70ef4183'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:57.778384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5126451e8e654a78ba9663ee70ef4183'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:57.779754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5126451e8e654a78ba9663ee70ef4183'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:57.795353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1415/2862] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:14:57 2022] DELETE /identity/v3/users/5126451e8e654a78ba9663ee70ef4183 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:14:57.799983 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:57.800472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:57.800596 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:57.821784 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:57.823354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:57.892137 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1448/2863] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:57.902438 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:57.904210 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:57.967188 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a006b-4085-425b-bfb7-fb133f36b258 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:57.970832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:57.971472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:57.971775 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:57.972044 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:57.980959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:57.982396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:58.003558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1416/2864] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:58.295855 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:58.297533 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.362040 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:58.363854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.389430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75ad7e-2e3f-41f7-bdf6-630ff737f4da service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:58.403310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:58.405639 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:58.405928 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:58.406204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] PATH_INFO: `/v3/projects/e679e7afe9de4010acc09828720aec62` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:58.407480 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e679e7afe9de4010acc09828720aec62)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:58.409930 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:58.435734 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148244e-1bf9-4bec-9221-a3622b981eaa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:58.441411 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:58.442024 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:58.442306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:58.442580 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:58.444381 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.445296 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:58.458240 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:58.459873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:58.470046 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e679e7afe9de4010acc09828720aec62', 'request_id': 'req-0604eda5-a899-486d-9338-4f5b1291e6c2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.470216 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1449/2865] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:58.470639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:58.472897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.476622 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e679e7afe9de4010acc09828720aec62', 'request_id': 'req-0604eda5-a899-486d-9338-4f5b1291e6c2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.488130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b87ad-76c0-425b-a9da-9d0040045f3d service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:58.490391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e679e7afe9de4010acc09828720aec62', 'request_id': 'req-0604eda5-a899-486d-9338-4f5b1291e6c2'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.507385 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] The token cache is being invalidate because project e679e7afe9de4010acc09828720aec62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:14:58.507892 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.509025 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.510780 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1417/2866] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:14:58 2022] DELETE /identity/v3/projects/e679e7afe9de4010acc09828720aec62 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:58.514430 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:58.515855 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.516123 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:58.547304 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:58.548129 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:58.548405 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:58.548725 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] PATH_INFO: `/v3/users/b339e5581cf044d9a55eb85561a58106` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:58.550409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b339e5581cf044d9a55eb85561a58106)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:58.552354 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:58.593153 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604eda5-a899-486d-9338-4f5b1291e6c2 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:58.598427 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:58.599329 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:58.599754 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:58.600039 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] PATH_INFO: `/v3/users/d0aa28b7705a418cac920c95a025ce5d` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:58.600335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b339e5581cf044d9a55eb85561a58106'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.601201 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0aa28b7705a418cac920c95a025ce5d)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:58.603155 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:58.607982 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b339e5581cf044d9a55eb85561a58106'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.620700 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b339e5581cf044d9a55eb85561a58106'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.621433 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b339e5581cf044d9a55eb85561a58106'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.622057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b339e5581cf044d9a55eb85561a58106'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.637258 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1450/2867] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:14:58 2022] DELETE /identity/v3/users/b339e5581cf044d9a55eb85561a58106 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:58.648194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:58.649598 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.657914 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0aa28b7705a418cac920c95a025ce5d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.659128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0aa28b7705a418cac920c95a025ce5d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.666286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0aa28b7705a418cac920c95a025ce5d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.680120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0aa28b7705a418cac920c95a025ce5d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.681013 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0aa28b7705a418cac920c95a025ce5d'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:58.694263 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1418/2868] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:14:58 2022] DELETE /identity/v3/users/d0aa28b7705a418cac920c95a025ce5d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:14:58.713088 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83779eb5-56c7-40fc-8223-86c409f76b17 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:58.764779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:58.765650 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:58.766016 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:58.766461 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:58.768640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.788670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:58.837030 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:58.838499 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:58.850404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1451/2869] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:14:58.978810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:58.980344 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.996096 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae387ba3-9f92-45bc-bca7-75856e39a996 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.012310 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.012973 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.013499 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.013812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] PATH_INFO: `/v3/projects/0b849b3f19a543eca4b8b9c85de90d9f` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.015142 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b849b3f19a543eca4b8b9c85de90d9f)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.017094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.038211 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.039781 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.053728 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6ee97-05d4-4694-863c-c4dcc80b749c service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.059339 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.059949 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.060261 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.060503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] PATH_INFO: `/v3/projects/f68efb39c9f649d6a0e2bc424b418cbd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.061669 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68efb39c9f649d6a0e2bc424b418cbd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.063795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.068773 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:59.090277 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b849b3f19a543eca4b8b9c85de90d9f', 'request_id': 'req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.098317 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b849b3f19a543eca4b8b9c85de90d9f', 'request_id': 'req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.106498 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b849b3f19a543eca4b8b9c85de90d9f', 'request_id': 'req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.115560 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68efb39c9f649d6a0e2bc424b418cbd', 'request_id': 'req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.117143 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] The token cache is being invalidate because project 0b849b3f19a543eca4b8b9c85de90d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:14:59.117422 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.118053 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e9ffe615-6901-40f0-b4eb-c2d0a0a963e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.119563 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1419/2870] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:14:58 2022] DELETE /identity/v3/projects/0b849b3f19a543eca4b8b9c85de90d9f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:59.122198 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68efb39c9f649d6a0e2bc424b418cbd', 'request_id': 'req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.125599 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905965d2-e3c7-451a-9998-d8463ed7ed14 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.125884 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905965d2-e3c7-451a-9998-d8463ed7ed14 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.126159 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905965d2-e3c7-451a-9998-d8463ed7ed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.128747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68efb39c9f649d6a0e2bc424b418cbd', 'request_id': 'req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.142378 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] The token cache is being invalidate because project f68efb39c9f649d6a0e2bc424b418cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:14:59.143296 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.143957 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.145472 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1452/2871] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:14:59 2022] DELETE /identity/v3/projects/f68efb39c9f649d6a0e2bc424b418cbd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:14:59.150496 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.150676 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-905965d2-e3c7-451a-9998-d8463ed7ed14 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:59.151807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.151943 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-905965d2-e3c7-451a-9998-d8463ed7ed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.215820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1420/2872] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:59.217788 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e9da-ba2d-4b03-8d3b-3e0a52bed995 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.223229 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.223832 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.224107 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.224227 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357c61-a3e1-4a1b-85f5-9a1363e07594 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.224380 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] PATH_INFO: `/v3/users/0752c7b0aa574a6ab87af239daeb178a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.224489 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357c61-a3e1-4a1b-85f5-9a1363e07594 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.224680 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357c61-a3e1-4a1b-85f5-9a1363e07594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.225514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] RBAC: Authorizing `identity:delete_user(user_id=0752c7b0aa574a6ab87af239daeb178a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.227415 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.243870 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-1c357c61-a3e1-4a1b-85f5-9a1363e07594 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:59.245068 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-1c357c61-a3e1-4a1b-85f5-9a1363e07594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.253440 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1421/2873] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:59.257903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357c61-a3e1-4a1b-85f5-9a1363e07594 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.267243 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357c61-a3e1-4a1b-85f5-9a1363e07594 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.269720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:59.272571 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.273192 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.273469 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.273743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.274375 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:14:59.274675 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.276306 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.276934 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0752c7b0aa574a6ab87af239daeb178a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.286208 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0752c7b0aa574a6ab87af239daeb178a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.290382 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1422/2874] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:14:59.294777 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.299237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0752c7b0aa574a6ab87af239daeb178a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.299971 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0752c7b0aa574a6ab87af239daeb178a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.300622 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0752c7b0aa574a6ab87af239daeb178a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.304957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb378a9-aa6d-4334-b59b-62b204cc5499 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.311211 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.311820 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.312101 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.312373 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.312996 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.314424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1453/2875] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:14:59 2022] DELETE /identity/v3/users/0752c7b0aa574a6ab87af239daeb178a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:59.315085 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.343841 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1423/2876] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:14:59 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:14:59.348756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.350284 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.416679 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb96d-4676-4004-ab73-1a133d1badab admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.472776 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.473448 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.473670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.473943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.474572 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.476518 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.495374 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.498384 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.505571 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1454/2877] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:14:59 2022] POST /identity/v3/users => generated 305 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:14:59.506633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.513620 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073aad-49a0-4ce2-8612-b3f3f0725b6c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.518668 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.519033 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2231c-6a3d-4acf-8af4-d888e6cb3cde admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.519335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.519806 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.520036 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] PATH_INFO: `/v3/projects/811c36b765454ac28ed99a50c3a801c5` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.521146 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] RBAC: Authorizing `identity:delete_project(project_id=811c36b765454ac28ed99a50c3a801c5)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.523038 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.525430 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.526468 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.526935 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.527210 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.528124 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.530035 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.540922 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1455/2878] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:14:59.544816 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.555527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb335-0515-4352-abd8-9469ac09ecf1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.562307 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.562927 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.563209 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.563516 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.564324 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.566772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.578365 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1456/2879] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:14:59.582258 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811c36b765454ac28ed99a50c3a801c5', 'request_id': 'req-e75db9a6-8a07-4559-8d47-32e7b741502b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.583443 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.592272 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811c36b765454ac28ed99a50c3a801c5', 'request_id': 'req-e75db9a6-8a07-4559-8d47-32e7b741502b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.593731 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16572-46aa-42d1-84e8-5960ced4f0c0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.601493 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811c36b765454ac28ed99a50c3a801c5', 'request_id': 'req-e75db9a6-8a07-4559-8d47-32e7b741502b'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.606705 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:59.616577 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] The token cache is being invalidate because project 811c36b765454ac28ed99a50c3a801c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:14:59.617327 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.620280 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-e75db9a6-8a07-4559-8d47-32e7b741502b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:14:59.622949 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1424/2880] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:14:59 2022] DELETE /identity/v3/projects/811c36b765454ac28ed99a50c3a801c5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:59.628933 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:59.668235 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.669128 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.669409 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.669682 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] PATH_INFO: `/v3/projects/731bdf8b241b4207903dc3985361995a/users/c570389a30364725ba7196af2674edaa/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.686825 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] RBAC: Authorizing `identity:create_grant(project_id=731bdf8b241b4207903dc3985361995a, user_id=c570389a30364725ba7196af2674edaa, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.690290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.704612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1457/2881] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:14:59 2022] PUT /identity/v3/projects/731bdf8b241b4207903dc3985361995a/users/c570389a30364725ba7196af2674edaa/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:14:59.715857 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef50428-ad46-4a0f-b917-e96c7b7dc766 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.716137 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef50428-ad46-4a0f-b917-e96c7b7dc766 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.716629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef50428-ad46-4a0f-b917-e96c7b7dc766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.748308 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-aef50428-ad46-4a0f-b917-e96c7b7dc766 None None] MFA Rules not processed for user `c570389a30364725ba7196af2674edaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:59.749573 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-aef50428-ad46-4a0f-b917-e96c7b7dc766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.821764 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1425/2882] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:14:59 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:59.826490 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.828305 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.893930 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faafc36-fa26-4449-9432-c1ec4352183d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.898630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.899237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.899552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.899828 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.900471 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.902402 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.929260 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1458/2883] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:14:59 2022] POST /identity/v3/projects => generated 407 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:14:59.933604 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef50428-ad46-4a0f-b917-e96c7b7dc766 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:59.944695 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef50428-ad46-4a0f-b917-e96c7b7dc766 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.999353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.999922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.000203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.000479 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.001114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:00.003644 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:00.023405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1426/2884] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:14:59 2022] POST /identity/v3/users => generated 310 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:15:00.027306 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:00.037450 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ba1ad-76f9-4fd9-9c03-cbe6d21dd2f9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:00.042052 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:00.042644 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.042924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.043196 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.043873 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:00.045456 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:00.056479 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1459/2885] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:15:00.060433 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:00.070963 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4da5b9-0628-4d55-951d-e8118d6c1b61 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:00.075629 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:00.076271 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.076555 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.076829 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] PATH_INFO: `/v3/projects/fbcbc6330ef6423ebb555069a7b1184e/users/28510173c19d4043be739a88523918bd/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.091163 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbcbc6330ef6423ebb555069a7b1184e, user_id=28510173c19d4043be739a88523918bd, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:00.093826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:00.105659 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1427/2886] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:15:00 2022] PUT /identity/v3/projects/fbcbc6330ef6423ebb555069a7b1184e/users/28510173c19d4043be739a88523918bd/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:00.109427 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:00.119815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abedcb-2af3-489d-9068-223bcd429428 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:00.175865 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40407ac5-a0bf-40b6-8f51-6451a6126a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:00.176464 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407ac5-a0bf-40b6-8f51-6451a6126a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.176741 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407ac5-a0bf-40b6-8f51-6451a6126a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.177006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407ac5-a0bf-40b6-8f51-6451a6126a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.177630 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40407ac5-a0bf-40b6-8f51-6451a6126a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:00.179194 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40407ac5-a0bf-40b6-8f51-6451a6126a5c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:00.189263 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1460/2887] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:15:00.193457 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:00.203752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50881445-5f4d-4b0e-88bc-aa58a42ec973 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:00.208449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:00.209117 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.209391 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.209656 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] PATH_INFO: `/v3/projects/fbcbc6330ef6423ebb555069a7b1184e/users/28510173c19d4043be739a88523918bd/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.211632 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbcbc6330ef6423ebb555069a7b1184e, user_id=28510173c19d4043be739a88523918bd, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:00.214225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:00.224846 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1428/2888] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:15:00 2022] PUT /identity/v3/projects/fbcbc6330ef6423ebb555069a7b1184e/users/28510173c19d4043be739a88523918bd/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:00.235610 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616a7f-01e5-46b7-a55c-56c9e2ad401e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.235899 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616a7f-01e5-46b7-a55c-56c9e2ad401e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.236172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616a7f-01e5-46b7-a55c-56c9e2ad401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.263693 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c7616a7f-01e5-46b7-a55c-56c9e2ad401e None None] MFA Rules not processed for user `28510173c19d4043be739a88523918bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:00.264897 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c7616a7f-01e5-46b7-a55c-56c9e2ad401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:00.276215 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:00.332503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1461/2889] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:00.343810 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:00.345472 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:00.387309 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:00.403259 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c5982-19b8-4acc-98be-16f17868cc8e admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:00.407213 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:00.407882 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.408191 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.409012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.418197 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:00.419732 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:00.429772 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1429/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:00.579301 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7616a7f-01e5-46b7-a55c-56c9e2ad401e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:00.589589 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7616a7f-01e5-46b7-a55c-56c9e2ad401e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:00.593202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd41fa-7db2-4af3-8cb6-47cb54d25325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:00.593795 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd41fa-7db2-4af3-8cb6-47cb54d25325 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.594076 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd41fa-7db2-4af3-8cb6-47cb54d25325 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.594350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd41fa-7db2-4af3-8cb6-47cb54d25325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.649594 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd41fa-7db2-4af3-8cb6-47cb54d25325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:00.651039 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd41fa-7db2-4af3-8cb6-47cb54d25325 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:00.660362 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1462/2891] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:00 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:07.967673 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:08.029545 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156ffd9-8ea9-4c39-a45d-c51a8a7ca939 service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:08.034081 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:08.034701 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.034981 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.035255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] PATH_INFO: `/v3/users/c570389a30364725ba7196af2674edaa` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.037014 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c570389a30364725ba7196af2674edaa)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:08.038911 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:08.085396 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c570389a30364725ba7196af2674edaa'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.085965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c570389a30364725ba7196af2674edaa'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.092965 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c570389a30364725ba7196af2674edaa'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.105235 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c570389a30364725ba7196af2674edaa'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.105859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c570389a30364725ba7196af2674edaa'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.118043 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1430/2892] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:15:07 2022] DELETE /identity/v3/users/c570389a30364725ba7196af2674edaa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:08.122654 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.123174 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.123292 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.141924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:08.143131 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:08.200393 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1463/2893] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:08.210484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:08.211916 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:08.268856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21dfa1-c779-4ca5-9283-0c0ece8d8dc4 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:08.272979 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:08.273574 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.273856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.274121 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.282842 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:08.284282 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:08.294255 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1431/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:08.565457 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:08.567475 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:08.579515 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34faf-e017-4e03-8fc3-134792c99405 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:08.585332 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:08.586005 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.586281 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.586552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] PATH_INFO: `/v3/projects/731bdf8b241b4207903dc3985361995a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.587720 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731bdf8b241b4207903dc3985361995a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:08.589719 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:08.631766 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:08.635880 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731bdf8b241b4207903dc3985361995a', 'request_id': 'req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.642426 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731bdf8b241b4207903dc3985361995a', 'request_id': 'req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.648612 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731bdf8b241b4207903dc3985361995a', 'request_id': 'req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.660257 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] The token cache is being invalidate because project 731bdf8b241b4207903dc3985361995a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:15:08.660545 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.661202 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.662702 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1464/2895] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:15:08 2022] DELETE /identity/v3/projects/731bdf8b241b4207903dc3985361995a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:08.666809 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:08.671643 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:08.678471 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebda51-69ad-48a8-8942-e7a035dc7fea service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:08.736506 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:08.737131 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.737405 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.737674 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] PATH_INFO: `/v3/users/28510173c19d4043be739a88523918bd` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.739275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28510173c19d4043be739a88523918bd)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:08.741234 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:08.785740 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28510173c19d4043be739a88523918bd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.786404 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28510173c19d4043be739a88523918bd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.793895 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28510173c19d4043be739a88523918bd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.806204 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28510173c19d4043be739a88523918bd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.806854 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6d509b40-bd0e-4552-ab77-2eb4488cb2b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28510173c19d4043be739a88523918bd'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:08.820743 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1432/2896] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:15:08 2022] DELETE /identity/v3/users/28510173c19d4043be739a88523918bd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:09.084643 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.086097 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:09.104647 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc4089-9d34-4293-bb2b-29603ddbdfb6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.161960 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.162603 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.162779 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.163047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] PATH_INFO: `/v3/projects/fbcbc6330ef6423ebb555069a7b1184e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.164166 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbcbc6330ef6423ebb555069a7b1184e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.166031 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.212259 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbcbc6330ef6423ebb555069a7b1184e', 'request_id': 'req-6563a909-5605-4ab1-86b0-45cd2641b786'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:09.218243 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbcbc6330ef6423ebb555069a7b1184e', 'request_id': 'req-6563a909-5605-4ab1-86b0-45cd2641b786'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:09.224279 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbcbc6330ef6423ebb555069a7b1184e', 'request_id': 'req-6563a909-5605-4ab1-86b0-45cd2641b786'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:09.236565 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] The token cache is being invalidate because project fbcbc6330ef6423ebb555069a7b1184e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:15:09.236857 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:09.237480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6563a909-5605-4ab1-86b0-45cd2641b786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:09.239632 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1465/2897] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:15:09 2022] DELETE /identity/v3/projects/fbcbc6330ef6423ebb555069a7b1184e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:09.244443 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545efffb-53f9-4403-a8ed-280f8236a51a None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.244942 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545efffb-53f9-4403-a8ed-280f8236a51a None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.245049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545efffb-53f9-4403-a8ed-280f8236a51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.264127 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-545efffb-53f9-4403-a8ed-280f8236a51a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:09.265349 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-545efffb-53f9-4403-a8ed-280f8236a51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:09.323633 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1433/2898] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:15:09.331414 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a756b-9d6b-4362-9aa6-b29508b7aa6a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.331744 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a756b-9d6b-4362-9aa6-b29508b7aa6a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.332047 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a756b-9d6b-4362-9aa6-b29508b7aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.351616 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7b9a756b-9d6b-4362-9aa6-b29508b7aa6a None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:09.352809 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b9a756b-9d6b-4362-9aa6-b29508b7aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:09.360568 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1466/2899] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:09.365125 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545efffb-53f9-4403-a8ed-280f8236a51a None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.374635 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545efffb-53f9-4403-a8ed-280f8236a51a None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.379112 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.379721 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.380000 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.380275 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.380903 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:15:09.381198 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.382759 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.394032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1434/2900] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:15:09.398130 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a756b-9d6b-4362-9aa6-b29508b7aa6a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.407349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a756b-9d6b-4362-9aa6-b29508b7aa6a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.412757 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.413295 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.413574 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.413842 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.414476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.416429 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.443527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1467/2901] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:15:09 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:15:09.447693 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.457383 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083004d-6002-4a94-bba9-f21248d30d7d admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.462394 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:09.519418 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.520583 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.520860 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.521128 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.521787 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.523704 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.545203 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1435/2902] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:15:09 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:15:09.549342 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.559805 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece66173-bf3f-4b19-935a-c1f4d044e8a8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.564368 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.564959 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.565239 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.565509 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.566167 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.567847 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.577543 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1468/2903] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:15:09.581490 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.591692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746ac1-80e2-43c5-a230-2c6bb267c430 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.596208 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdcd37-62e9-46af-9550-2e4a2aebe4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.596789 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdcd37-62e9-46af-9550-2e4a2aebe4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.597065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdcd37-62e9-46af-9550-2e4a2aebe4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.597331 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdcd37-62e9-46af-9550-2e4a2aebe4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.597957 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdcd37-62e9-46af-9550-2e4a2aebe4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.600401 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdcd37-62e9-46af-9550-2e4a2aebe4d3 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.610739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1436/2904] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:15:09.614640 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.624527 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf358a-db93-4806-929e-fab47d87538c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.629109 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.629743 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.630019 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.630290 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] PATH_INFO: `/v3/projects/421e03f2f6a84acc80890e38006f4225/users/db3c8020b17c4fc1b39e003e9a61b17c/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.644445 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421e03f2f6a84acc80890e38006f4225, user_id=db3c8020b17c4fc1b39e003e9a61b17c, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.647587 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.655639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:09.662708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1469/2905] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:15:09 2022] PUT /identity/v3/projects/421e03f2f6a84acc80890e38006f4225/users/db3c8020b17c4fc1b39e003e9a61b17c/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:09.672865 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075a4ed-535f-489a-91fd-3922702002d8 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.673115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075a4ed-535f-489a-91fd-3922702002d8 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.673386 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075a4ed-535f-489a-91fd-3922702002d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.700470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-8075a4ed-535f-489a-91fd-3922702002d8 None None] MFA Rules not processed for user `db3c8020b17c4fc1b39e003e9a61b17c`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:09.701692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-8075a4ed-535f-489a-91fd-3922702002d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:09.766449 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1437/2906] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:09.771537 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.832172 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50344509-8482-433d-9065-2bf0404e62a5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.839360 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.839998 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.840351 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.840850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.841974 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.843917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.869883 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1470/2907] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:15:09 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:15:09.873959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075a4ed-535f-489a-91fd-3922702002d8 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.885003 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075a4ed-535f-489a-91fd-3922702002d8 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.940120 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.940648 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.940927 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.941199 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.942303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.944256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.964760 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1438/2908] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:15:09 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:15:09.968851 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:09.979226 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416384a-aa20-43f1-8563-61b25a5bcc14 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:09.983791 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:09.984379 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:09.984656 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:09.984924 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:09.985548 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:09.987121 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.997503 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1471/2909] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:15:10.001388 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:10.011660 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eef31-ab61-4c40-9407-a9be18ea992c admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:10.016237 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:10.016877 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:10.017158 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:10.017432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] PATH_INFO: `/v3/projects/4366aa3b5dbb4d8d96b207acb3519be4/users/30657016f730433ea0651f1a26d37189/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:10.019601 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:10.033679 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4366aa3b5dbb4d8d96b207acb3519be4, user_id=30657016f730433ea0651f1a26d37189, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:10.036398 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:10.048907 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1439/2910] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:15:10 2022] PUT /identity/v3/projects/4366aa3b5dbb4d8d96b207acb3519be4/users/30657016f730433ea0651f1a26d37189/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:10.052721 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:10.062895 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d8b66-8900-46b8-918f-23431cae8bf3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:10.117383 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa645-d422-4b22-95b0-a80984a67f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:10.117988 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa645-d422-4b22-95b0-a80984a67f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:10.118268 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa645-d422-4b22-95b0-a80984a67f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:10.118538 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa645-d422-4b22-95b0-a80984a67f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:10.119165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa645-d422-4b22-95b0-a80984a67f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:10.120777 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa645-d422-4b22-95b0-a80984a67f5f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:10.130904 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1472/2911] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:15:10.134959 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:10.146719 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c65d9-28de-4ae1-98b4-23e62fb5af24 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:10.151400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:10.152094 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:10.152378 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:10.152655 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] PATH_INFO: `/v3/projects/4366aa3b5dbb4d8d96b207acb3519be4/users/30657016f730433ea0651f1a26d37189/roles/67db52226a7a4fd0b469e8de2c12b8e3` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:10.154578 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4366aa3b5dbb4d8d96b207acb3519be4, user_id=30657016f730433ea0651f1a26d37189, role_id=67db52226a7a4fd0b469e8de2c12b8e3)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:10.157250 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:10.167369 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1440/2912] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:15:10 2022] PUT /identity/v3/projects/4366aa3b5dbb4d8d96b207acb3519be4/users/30657016f730433ea0651f1a26d37189/roles/67db52226a7a4fd0b469e8de2c12b8e3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:10.231708 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7156d87-df0a-433b-9d4e-1ea9af92817b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:10.232247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7156d87-df0a-433b-9d4e-1ea9af92817b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:10.232247 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7156d87-df0a-433b-9d4e-1ea9af92817b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:10.260639 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f7156d87-df0a-433b-9d4e-1ea9af92817b None None] MFA Rules not processed for user `30657016f730433ea0651f1a26d37189`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:10.261887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f7156d87-df0a-433b-9d4e-1ea9af92817b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:10.318404 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:10.329242 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1473/2913] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:10.340835 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:10.342663 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:10.400672 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a254d8-7e61-4dc6-94d5-b008f98020f1 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:10.404357 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:10.404953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:10.405235 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:10.405510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:10.416302 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:10.417728 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:10.427115 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1441/2914] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:10.509444 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7156d87-df0a-433b-9d4e-1ea9af92817b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:10.519011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7156d87-df0a-433b-9d4e-1ea9af92817b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:10.522607 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40692f51-ee63-4ada-918c-fcc8e1351a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:10.523200 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692f51-ee63-4ada-918c-fcc8e1351a0b service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:10.523476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692f51-ee63-4ada-918c-fcc8e1351a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:10.523772 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692f51-ee63-4ada-918c-fcc8e1351a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:10.579066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40692f51-ee63-4ada-918c-fcc8e1351a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:10.580514 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40692f51-ee63-4ada-918c-fcc8e1351a0b service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:10.589321 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1474/2915] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:15.260127 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:15.321748 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc014f-39f8-4efc-8a00-b09432b1c15a service cinder] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:15.326134 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:15.326746 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:15.327020 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:15.327379 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] PATH_INFO: `/v3/users/db3c8020b17c4fc1b39e003e9a61b17c` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:15.328510 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3c8020b17c4fc1b39e003e9a61b17c)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:15.330417 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:15.373739 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3c8020b17c4fc1b39e003e9a61b17c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.374376 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3c8020b17c4fc1b39e003e9a61b17c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.381756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3c8020b17c4fc1b39e003e9a61b17c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.393135 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3c8020b17c4fc1b39e003e9a61b17c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.393752 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3c8020b17c4fc1b39e003e9a61b17c'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.395353 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:15.408593 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1442/2916] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:15:15 2022] DELETE /identity/v3/users/db3c8020b17c4fc1b39e003e9a61b17c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:15.413313 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:15.413838 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:15.413965 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:15.433276 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:15.434480 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:15.492887 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1475/2917] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:15.502516 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:15.503953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:15.561425 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadaa6f-6434-40d7-b0da-ef7fd1dd3fd0 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:15.565102 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:15.565689 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:15.565970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:15.566242 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:15.575727 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:15.577083 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:15.586305 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1443/2918] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:15.857614 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:15.859057 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:15.872327 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01172f0-3943-471d-a26c-5a4abbfc31b7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:15.877006 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:15.877637 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:15.877918 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:15.878188 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] PATH_INFO: `/v3/projects/421e03f2f6a84acc80890e38006f4225` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:15.879349 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=421e03f2f6a84acc80890e38006f4225)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:15.881335 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:15.929985 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421e03f2f6a84acc80890e38006f4225', 'request_id': 'req-9c5939d6-2317-4c40-94f0-25344a55f67e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.936253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421e03f2f6a84acc80890e38006f4225', 'request_id': 'req-9c5939d6-2317-4c40-94f0-25344a55f67e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.941787 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421e03f2f6a84acc80890e38006f4225', 'request_id': 'req-9c5939d6-2317-4c40-94f0-25344a55f67e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.954010 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] The token cache is being invalidate because project 421e03f2f6a84acc80890e38006f4225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:15:15.954442 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.955113 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:15.956765 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1476/2919] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:15:15 2022] DELETE /identity/v3/projects/421e03f2f6a84acc80890e38006f4225 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:15.960923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:15.970316 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfc8b4-3f7f-455a-9f51-82dee94a8f93 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.025453 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.026066 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.026345 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.026619 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] PATH_INFO: `/v3/users/30657016f730433ea0651f1a26d37189` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.027766 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30657016f730433ea0651f1a26d37189)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.029686 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.073826 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30657016f730433ea0651f1a26d37189'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.074466 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30657016f730433ea0651f1a26d37189'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.081166 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30657016f730433ea0651f1a26d37189'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.093065 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30657016f730433ea0651f1a26d37189'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.093707 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-43e9a452-1591-40ad-a538-ba4a42ec50e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30657016f730433ea0651f1a26d37189'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.107011 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1444/2920] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:15:15 2022] DELETE /identity/v3/users/30657016f730433ea0651f1a26d37189 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:16.369135 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:16.370590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:16.385225 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5939d6-2317-4c40-94f0-25344a55f67e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.398213 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:16.445326 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.446029 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.446212 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.446476 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] PATH_INFO: `/v3/projects/4366aa3b5dbb4d8d96b207acb3519be4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.447670 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4366aa3b5dbb4d8d96b207acb3519be4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.449590 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.494253 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4366aa3b5dbb4d8d96b207acb3519be4', 'request_id': 'req-73c8a042-76fb-45bb-8a10-4bef62ede3d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.500237 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4366aa3b5dbb4d8d96b207acb3519be4', 'request_id': 'req-73c8a042-76fb-45bb-8a10-4bef62ede3d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.506140 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4366aa3b5dbb4d8d96b207acb3519be4', 'request_id': 'req-73c8a042-76fb-45bb-8a10-4bef62ede3d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.518850 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] The token cache is being invalidate because project 4366aa3b5dbb4d8d96b207acb3519be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:15:16.519139 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.519785 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-73c8a042-76fb-45bb-8a10-4bef62ede3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:16.521796 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1477/2921] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:15:16 2022] DELETE /identity/v3/projects/4366aa3b5dbb4d8d96b207acb3519be4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:16.528297 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc43656-5ea1-42b0-a581-4f9dc00bd2be None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.529006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc43656-5ea1-42b0-a581-4f9dc00bd2be None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.529006 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc43656-5ea1-42b0-a581-4f9dc00bd2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.548902 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7cc43656-5ea1-42b0-a581-4f9dc00bd2be None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:16.550104 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7cc43656-5ea1-42b0-a581-4f9dc00bd2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:16.558670 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:16.612207 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1445/2922] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:15:16.619871 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea2b73-e733-49b8-b47a-c6c93fc07d5f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.620155 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea2b73-e733-49b8-b47a-c6c93fc07d5f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.620425 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea2b73-e733-49b8-b47a-c6c93fc07d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.641651 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d8ea2b73-e733-49b8-b47a-c6c93fc07d5f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:16.642860 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d8ea2b73-e733-49b8-b47a-c6c93fc07d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:16.651270 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1478/2923] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:16.655432 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc43656-5ea1-42b0-a581-4f9dc00bd2be None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:16.665114 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc43656-5ea1-42b0-a581-4f9dc00bd2be None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.669642 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.670232 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.670512 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.670784 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.671416 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:15:16.671749 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.673332 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.684972 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1446/2924] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 23:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:15:16.689197 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea2b73-e733-49b8-b47a-c6c93fc07d5f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:16.698815 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea2b73-e733-49b8-b47a-c6c93fc07d5f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.703377 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.703995 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.704272 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.704539 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.705165 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.707040 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.733852 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1479/2925] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 23:15:16 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:15:16.738177 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:16.748045 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9c29e-18e1-430c-8dab-7fa5d5b0e499 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.803286 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.803923 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.804205 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.804474 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] PATH_INFO: `/v3/users` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.805105 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.807257 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.828303 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1447/2926] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 23:15:16 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:15:16.832613 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:16.843540 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac401ef-e527-4ec2-989a-62f5038c1add admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.848138 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.848756 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.849037 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.849308 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.849943 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.851554 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.865576 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1480/2927] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:15:16.869859 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:16.882733 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c66634-e3cc-4753-83a7-ec8a0648e1ad admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.890060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3d3fe-10cd-40ab-9ce4-e8c79616d498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.890669 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3d3fe-10cd-40ab-9ce4-e8c79616d498 admin admin] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.890953 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3d3fe-10cd-40ab-9ce4-e8c79616d498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.891225 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3d3fe-10cd-40ab-9ce4-e8c79616d498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.891891 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3d3fe-10cd-40ab-9ce4-e8c79616d498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.893484 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3d3fe-10cd-40ab-9ce4-e8c79616d498 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.904969 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1448/2928] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 23:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:15:16.909274 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:16.919575 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998a0c-c849-4162-b782-6d394fdf9282 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:16.924399 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:16.925073 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.925350 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.925623 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] PATH_INFO: `/v3/projects/86f28b44fa6a45d08665e8350153f5ad/users/48b08570cd7140788aab42ae538bea39/roles/01c59f6e5c2b4b7a8eac21824e49fb9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.943925 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f28b44fa6a45d08665e8350153f5ad, user_id=48b08570cd7140788aab42ae538bea39, role_id=01c59f6e5c2b4b7a8eac21824e49fb9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:16.946552 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:16.958836 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1481/2929] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 23:15:16 2022] PUT /identity/v3/projects/86f28b44fa6a45d08665e8350153f5ad/users/48b08570cd7140788aab42ae538bea39/roles/01c59f6e5c2b4b7a8eac21824e49fb9d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:16.970584 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797081f-52dd-494e-b5b1-8eb67be02597 None None] REQUEST_METHOD: `POST` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:16.970876 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797081f-52dd-494e-b5b1-8eb67be02597 None None] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:16.971150 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797081f-52dd-494e-b5b1-8eb67be02597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:16.999012 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.auth.core [None req-7797081f-52dd-494e-b5b1-8eb67be02597 None None] MFA Rules not processed for user `48b08570cd7140788aab42ae538bea39`. Rule list: `[]` (Enabled: `True`). {{(pid=81404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:17.000256 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-7797081f-52dd-494e-b5b1-8eb67be02597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:17.064832 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1449/2930] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:17.076532 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:17.078227 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:17.122330 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:17.135469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f5ec8-19d1-49ae-a636-5b527cd74779 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:17.139043 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be595a-baba-4744-bbf5-0d9a70ea8b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56228e09c8b742b29e49c40d1a39870f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:17.139677 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be595a-baba-4744-bbf5-0d9a70ea8b21 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:17.139952 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be595a-baba-4744-bbf5-0d9a70ea8b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:17.140219 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be595a-baba-4744-bbf5-0d9a70ea8b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:17.148394 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be595a-baba-4744-bbf5-0d9a70ea8b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:17.149807 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be595a-baba-4744-bbf5-0d9a70ea8b21 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:17.158318 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1482/2931] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:17 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:24.195579 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797081f-52dd-494e-b5b1-8eb67be02597 None None] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:24.204948 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:24.258951 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797081f-52dd-494e-b5b1-8eb67be02597 None None] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:24.263397 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:24.264067 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:24.264335 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:24.264611 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] PATH_INFO: `/v3/users/48b08570cd7140788aab42ae538bea39` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:24.265730 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b08570cd7140788aab42ae538bea39)` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:24.267671 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:24.313109 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b08570cd7140788aab42ae538bea39'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.313756 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b08570cd7140788aab42ae538bea39'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.321041 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b08570cd7140788aab42ae538bea39'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.332612 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b08570cd7140788aab42ae538bea39'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.333812 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.notifications [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b08570cd7140788aab42ae538bea39'} {{(pid=81404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.345998 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1450/2932] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 23:15:24 2022] DELETE /identity/v3/users/48b08570cd7140788aab42ae538bea39 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:24.350521 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:24.351011 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:24.351118 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:24.369244 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] MFA Rules not processed for user `fa41a2a2137d4147a83f9f036ba03916`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:24.370424 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:24.428547 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1483/2933] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 23:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:24.438403 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:24.439856 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:24.495470 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eaa769-86e9-43af-b6ca-abc97e6b1946 admin admin] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:24.499060 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0e10b1ef7454167bd1c606ef4f70bf3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:24.499692 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:24.499967 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:24.500233 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:24.509558 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:24.510968 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:24.520079 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1451/2934] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:24.787223 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:24.788726 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:24.801392 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7cdc-4b6c-43a9-a5e9-17ebc797c38f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:24.807066 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa41a2a2137d4147a83f9f036ba03916', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8675cee43cc84599b492f8c81300cc82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:24.807747 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:24.808023 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:24.808298 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] PATH_INFO: `/v3/projects/86f28b44fa6a45d08665e8350153f5ad` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:24.809469 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f28b44fa6a45d08665e8350153f5ad)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:24.811914 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:24.857090 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f28b44fa6a45d08665e8350153f5ad', 'request_id': 'req-3cd9a850-78c3-45c2-ab33-e94c97288264'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.863003 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f28b44fa6a45d08665e8350153f5ad', 'request_id': 'req-3cd9a850-78c3-45c2-ab33-e94c97288264'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.869068 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f28b44fa6a45d08665e8350153f5ad', 'request_id': 'req-3cd9a850-78c3-45c2-ab33-e94c97288264'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.880633 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] The token cache is being invalidate because project 86f28b44fa6a45d08665e8350153f5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:15:24.880917 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.881546 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3cd9a850-78c3-45c2-ab33-e94c97288264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:15:24.882989 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1484/2935] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 23:15:24 2022] DELETE /identity/v3/projects/86f28b44fa6a45d08665e8350153f5ad => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:15:34.516032 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:34.877711 dv-cinder-852211-14-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:42.682922 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] Authenticating user token {{(pid=81404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:16:42.684400 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:42.753999 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285165e-b1f5-4ec8-8b3f-79702d7372c6 service neutron] Validating token access rules against request {{(pid=81404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:16:42.812179 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72110430-77a8-413e-af92-38ef9ecefb40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd9193fd644f4ec7a39d9c6ab8a1e525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632d504031cb466299331db2d9184912', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:16:42.812779 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72110430-77a8-413e-af92-38ef9ecefb40 service placement] REQUEST_METHOD: `GET` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:42.813058 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72110430-77a8-413e-af92-38ef9ecefb40 service placement] SCRIPT_NAME: `/identity` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:42.813330 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72110430-77a8-413e-af92-38ef9ecefb40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:42.815049 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.fernet_utils [None req-72110430-77a8-413e-af92-38ef9ecefb40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:42.825970 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72110430-77a8-413e-af92-38ef9ecefb40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:16:42.827556 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72110430-77a8-413e-af92-38ef9ecefb40 service placement] RBAC: Authorization granted {{(pid=81404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:16:42.837262 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: [pid: 81404|app: 0|req: 1452/2936] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 23:16:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:16:52.833048 dv-cinder-852211-14-iscsi devstack@keystone.service[81404]: DEBUG dbcounter [-] [81404] Writing DB stats keystone:SELECT=58 {{(pid=81404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}